├── blogs └── Simple-text-to-sql │ ├── sample-UI-for-market-research-assistant │ └── testfile │ ├── images │ ├── img-genai-sql-langchain.png │ └── img-genai-sql-langchain-overall-solution.png │ ├── mda-genai-blog-post.md │ ├── mda-llm-cfn.yml │ ├── mda_with_llm_langchain_smjumpstart_flant5xl.ipynb │ ├── mda_with_llm_langchain_byo_model_without_cloudformation.ipynb │ ├── mda_with_llm_langchain_byo_model.ipynb │ └── mda_text-to-sql_with_llm_langchain_bedrock.ipynb ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md └── CONTRIBUTING.md /blogs/Simple-text-to-sql/sample-UI-for-market-research-assistant/testfile: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/images/img-genai-sql-langchain.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-genai-datamesh/HEAD/blogs/Simple-text-to-sql/images/img-genai-sql-langchain.png -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/images/img-genai-sql-langchain-overall-solution.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-genai-datamesh/HEAD/blogs/Simple-text-to-sql/images/img-genai-sql-langchain-overall-solution.png -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/mda-genai-blog-post.md: -------------------------------------------------------------------------------- 1 | Modern Data Architecture interactions with Amazon SageMaker and LangChain 2 | ================ 3 | 4 | 5 | 6 | #### Use AWS Cloud Formation to create the solution stack 7 | 8 | We use AWS CloudFormation to create a SageMaker notebook called 9 | `xyz` and an IAM role called `xyz`. Choose 10 | **Launch Stack** for the Region you want to deploy resources to. All 11 | parameters needed by the CloudFormation template have default values 12 | already filled in. **This template takes 13 | about 5 minutes to complete**. 14 | 15 | 16 | After the stack is created successfully, navigate to the stack’s `Outputs` tab on the AWS CloudFormation console and note the values for 17 | `LLMAppAPIEndpoint`. We use those in the subsequent steps. 18 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Modern Data Architectures with Large Language Models 2 | 3 | This repository provides the source code for use cases where applications interact with Large Language Model (LLM) using Langchain framework and text to SQL. 4 | There are two modes demonstrated here: 5 | 6 | **Mode 1** - Use of LLMs with Langchain via Amazon Sagemaker Jumpstart. Amazon Sagemaker Jumpstart makes it simple to host LLMs as Sagemaker Endpoints enabling uses-cases for inference and embeddings generation. 7 | 8 | Refer to Notebooks - 9 | 10 | i. Path - blogs/Simple-text-to-sql/mda_with_llm_langchain_smjumpstart_flant5xl.ipynb 11 | 12 | Dependency - Run the cloudformation script (blogs/Simple-text-to-sql/mda-llm-cfn.yml ) prior to running the notebook. 13 | 14 | Notes - The notebook showcases the concept with one database source but can be easily extended to other data bases. Connection code is commented for some of the supported databases. For those databases, ensure AWS Glue Crawler is first run on the underlying data sources. 15 | 16 | **Mode 2** - Bring your own model api. In this case, provide API keys of your LLM to the Langchain framework. 17 | 18 | i. Path - blogs/Simple-text-to-sql/mda_with_llm_langchain_byo_model.ipynb 19 | 20 | Dependency - Run the cloudformation script (blogs/Simple-text-to-sql/mda-llm-cfn.yml ) prior to running the notebook. 21 | 22 | Notes - The notebook showcases the concept with one database source but can be easily extended to other data bases. Connection code is commented for some of the supported databases. For those databases, ensure AWS Glue Crawler is first run on the underlying data sources. 23 | 24 | ii. Path - blogs/Simple-text-to-sql/mda_with_llm_langchain_byo_model_wo_cloudformation.ipynb 25 | 26 | Dependency - Ensure the Sagemaker execution role has all the required privileges mentioned in teh notebook. No need to run the cloudformation acript. 27 | 28 | Notes - The notebook runs seamlessly for one data source. It can be easily extended to multiple data base sources. 29 | For the sake of showing multiple data sources, Weather API has been added. 30 | 31 | **Mode 3** - Use of LLMs with Langchain via Amazon Bedrock. 32 | 33 | TBD 34 | 35 | ## Repository structure 36 | The code in this repo is organized into the following sub-folders. 37 | 38 | ```├── README.md 39 | ├── blogs/ 40 | ├──────/Simple text to sql 41 | ├── workshop/ 42 | ├──────/TBD 43 | ├── images/ 44 | 45 | ## Security 46 | 47 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 48 | 49 | ## License 50 | 51 | This library is licensed under the MIT-0 License. See the LICENSE file. 52 | 53 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/mda-llm-cfn.yml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: 2010-09-09 2 | Description: Template to provision SageMaker Notebook for 'text to sql' example 3 | Parameters: 4 | DataBucketName: 5 | Type: String 6 | Description: "S3 bucket that holds data for workshop" 7 | Default: "Mention the name of the bucket that hosts data" 8 | SageMakerNotebookName: 9 | Default: aws-genai-mda-blog 10 | Type: String 11 | Description: Enter name of SageMaker Notebook instance. The notebook name must _not_ already exist in your AWS account/region. 12 | MinLength: 1 13 | MaxLength: 63 14 | AllowedPattern: ^[a-z0-9](-*[a-z0-9])* 15 | ConstraintDescription: Must be lowercase or numbers with a length of 1-63 characters. 16 | SageMakerIAMRole: 17 | Description: Name of IAM role that will be created by this cloud formation template. The role name must _not_ already exist in your AWS account. 18 | Type: String 19 | Default: "awsGenAIMDAblogIAMRole" 20 | CFNCrawlerName: 21 | Type: String 22 | Default: cfn-crawler-json 23 | CFNDatabaseName: 24 | Type: String 25 | Default: cfn_covid_lake 26 | CFNTablePrefixName: 27 | Type: String 28 | Default: cfn_ 29 | Resources: 30 | CodeRepository: 31 | Type: AWS::SageMaker::CodeRepository 32 | Properties: 33 | GitConfig: 34 | RepositoryUrl: https://github.com/aws-samples/amazon-sagemaker-genai-datamesh.git 35 | NotebookInstance: 36 | Type: AWS::SageMaker::NotebookInstance 37 | Properties: 38 | NotebookInstanceName: !Ref SageMakerNotebookName 39 | InstanceType: ml.t3.2xlarge 40 | RoleArn: !GetAtt Role.Arn 41 | DefaultCodeRepository: !GetAtt CodeRepository.CodeRepositoryName 42 | Role: 43 | Type: AWS::IAM::Role 44 | Properties: 45 | RoleName: !Ref SageMakerIAMRole 46 | Policies: 47 | - PolicyName: CustomNotebookSecretsAccess 48 | PolicyDocument: 49 | Version: 2012-10-17 50 | Statement: 51 | - Sid: ReadSecretFromSecretsManager 52 | Effect: Allow 53 | Action: 54 | - "secretsmanager:GetSecretValue" 55 | Resource: !Sub "arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:*" 56 | - PolicyName: s3_access_blog_genai_mda 57 | PolicyDocument: 58 | Version: 2012-10-17 59 | Statement: 60 | - Sid: PublicS3BucketReadAccess 61 | Effect: Allow 62 | Action: 63 | - "s3:GetObject" 64 | Resource: "arn:aws:s3:::*/**" 65 | - Sid: DestinationS3BucketWriteAccess 66 | Effect: Allow 67 | Action: 68 | - "s3:PutObject" 69 | Resource: !Join ["",["arn:aws:s3:::",!Ref DataBucketName,"/*"]] 70 | ManagedPolicyArns: 71 | - arn:aws:iam::aws:policy/AmazonSageMakerFullAccess 72 | - arn:aws:iam::aws:policy/AWSCloudFormationReadOnlyAccess 73 | - arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole 74 | - arn:aws:iam::aws:policy/AmazonAthenaFullAccess 75 | AssumeRolePolicyDocument: 76 | Version: 2012-10-17 77 | Statement: 78 | - Effect: Allow 79 | Principal: 80 | Service: 81 | - sagemaker.amazonaws.com 82 | Action: 83 | - 'sts:AssumeRole' 84 | LLMEndpoint: 85 | Type: "AWS::SageMaker::Endpoint" 86 | Properties: 87 | EndpointName: !Sub 88 | - 'aws-genai-mda-blog-flan-t5-xxl-endpoint-${RandomGUID}' 89 | - { RandomGUID: !Select [0, !Split ["-", !Select [2, !Split ["/", !Ref AWS::StackId ]]]] } 90 | EndpointConfigName: !GetAtt LLMEndpointConfig.EndpointConfigName 91 | LLMEndpointConfig: 92 | Type: "AWS::SageMaker::EndpointConfig" 93 | Properties: 94 | EndpointConfigName: !Sub 95 | - 'aws-genai-mda-blog-flan-t5-xxl-endpoint-${RandomGUID}' 96 | - { RandomGUID: !Select [0, !Split ["-", !Select [2, !Split ["/", !Ref AWS::StackId ]]]] } 97 | ProductionVariants: 98 | - InitialInstanceCount: 1 99 | InitialVariantWeight: 1.0 100 | InstanceType: "ml.g5.12xlarge" 101 | ModelName: !GetAtt LLMModel.ModelName 102 | VariantName: !GetAtt LLMModel.ModelName 103 | Metadata: 104 | cfn_nag: 105 | rules_to_suppress: 106 | - id: W1200 107 | reason: Solution does not have KMS encryption enabled by default 108 | LLMModel: 109 | Type: "AWS::SageMaker::Model" 110 | Properties: 111 | ModelName: !Sub 112 | - 'aws-genai-mda-blog-flan-t5-xxl-model-${RandomGUID}' 113 | - { RandomGUID: !Select [0, !Split ["-", !Select [2, !Split ["/", !Ref AWS::StackId ]]]] } 114 | PrimaryContainer: 115 | ModelDataUrl: !Sub "s3://jumpstart-cache-prod-${AWS::Region}/huggingface-infer/prepack/v1.0.1/infer-prepack-huggingface-text2text-flan-t5-xxl.tar.gz" 116 | Image: !Sub "763104351884.dkr.ecr.${AWS::Region}.amazonaws.com/pytorch-inference:1.12.0-gpu-py38" 117 | Environment: {"TS_DEFAULT_WORKERS_PER_MODEL": "1"} 118 | Mode: "SingleModel" 119 | ExecutionRoleArn: !GetAtt Role.Arn 120 | CFNRoleCovidLake: 121 | Type: AWS::IAM::Role 122 | Properties: 123 | AssumeRolePolicyDocument: 124 | Version: "2012-10-17" 125 | Statement: 126 | - 127 | Effect: "Allow" 128 | Principal: 129 | Service: 130 | - "glue.amazonaws.com" 131 | Action: 132 | - "sts:AssumeRole" 133 | Path: "/" 134 | Policies: 135 | - PolicyName: glueaccess_for_crawler 136 | PolicyDocument: 137 | Version: 2012-10-17 138 | Statement: 139 | - Sid: Readcrawlerresources 140 | Effect: Allow 141 | Action: 142 | - "glue:*" 143 | Resource: 144 | - !Join ["",[!Sub "arn:aws:glue:${AWS::Region}:${AWS::AccountId}:catalog"]] 145 | - !Join ["",[!Sub "arn:aws:glue:${AWS::Region}:${AWS::AccountId}:database/cfn_covid_lake"]] 146 | - !Join ["",[!Sub "arn:aws:glue:${AWS::Region}:${AWS::AccountId}:table/cfn_covid_lake/*"]] 147 | - PolicyName: s3_access_for_crawler 148 | PolicyDocument: 149 | Version: 2012-10-17 150 | Statement: 151 | - Sid: ReadS3Resources 152 | Effect: Allow 153 | Action: 154 | - "s3:PutObject" 155 | - "s3:GetObject" 156 | - "s3:PutBucketLogging" 157 | - "s3:ListBucket" 158 | - "s3:PutBucketVersioning" 159 | Resource: 160 | - !Join ["",["arn:aws:s3:::",!Ref DataBucketName] ] 161 | - !Join ["",["arn:aws:s3:::",!Ref DataBucketName,"/*"] ] 162 | - PolicyName: logaccess_for_crawler 163 | PolicyDocument: 164 | Version: 2012-10-17 165 | Statement: 166 | - Sid: ReadlogResources 167 | Effect: Allow 168 | Action: 169 | - "logs:CreateLogGroup" 170 | - "logs:CreateLogStream" 171 | - "logs:PutLogEvents" 172 | Resource: 173 | - !Join ["",[!Sub "arn:aws:glue:${AWS::Region}:${AWS::AccountId}:log-group:/aws-glue/crawlers*"]] 174 | # Create an AWS Glue database 175 | CFNDatabaseCovid: 176 | Type: AWS::Glue::Database 177 | Properties: 178 | CatalogId: !Ref AWS::AccountId 179 | DatabaseInput: 180 | Name: !Ref CFNDatabaseName 181 | Description: Database to hold tables for covid data 182 | #Create a crawler to crawl the workshop JSON data 183 | CFNCrawlerJSON: 184 | Type: AWS::Glue::Crawler 185 | Properties: 186 | Name: !Ref CFNCrawlerName 187 | Role: !GetAtt CFNRoleCovidLake.Arn 188 | #Classifiers: none, use the default classifier 189 | Description: AWS Glue crawler to crawl covid lake data 190 | #Schedule: none, use default run-on-demand 191 | DatabaseName: !Ref CFNDatabaseName 192 | Targets: 193 | S3Targets: 194 | # Public S3 bucket with the flights data 195 | - Path: !Join ["", ["s3://", !Ref "DataBucketName", "/covid-dataset/"]] 196 | TablePrefix: !Ref CFNTablePrefixName 197 | SchemaChangePolicy: 198 | UpdateBehavior: "UPDATE_IN_DATABASE" 199 | DeleteBehavior: "LOG" 200 | Configuration: "{\"Version\":1.0,\"CrawlerOutput\":{\"Partitions\":{\"AddOrUpdateBehavior\":\"InheritFromTable\"},\"Tables\":{\"AddOrUpdateBehavior\":\"MergeNewColumns\"}}}" 201 | Outputs: 202 | GlueCrawlerName: 203 | Description: Glue Crawler Name 204 | Value: !Ref CFNCrawlerName 205 | SageMakerNotebookURL: 206 | Description: SageMaker Notebook Instance 207 | Value: !Join 208 | - '' 209 | - - !Sub 'https://console.aws.amazon.com/sagemaker/home?region=${AWS::Region}#/notebook-instances/openNotebook/' 210 | - !GetAtt NotebookInstance.NotebookInstanceName 211 | - '?view=classic' 212 | LLMEndpointName: 213 | Description: Name of the LLM endpoint 214 | Value: !GetAtt LLMEndpoint.EndpointName 215 | Region: 216 | Description: Deployed Region 217 | Value: !Ref AWS::Region 218 | -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/mda_with_llm_langchain_smjumpstart_flant5xl.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0855b561", 6 | "metadata": {}, 7 | "source": [ 8 | "# Integrate Modern Data Architectures with Generative AI and interact using prompts for querying SQL databases & APIs" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "26105729-b3e3-42d0-a583-8446fff89277", 14 | "metadata": {}, 15 | "source": [ 16 | "This notebook demonstrates how **_large language models, such as Flan T5 XL,accessible via SamgeMaker JumpStart_** interact with AWS databases, data stores, and third-party data warehousing solutions like Snowflake. We showcase this interaction 1) by generating and running SQL queries, and 2) making requests to API endpoints. We achieve all of this by using the LangChain framework, which allows the language model to interact with its environment and connect with other sources of data. The LangChain framework operates based on the following principles: calling out to a language model, being data-aware, and being agentic. Our notebook focuses on establishing database connections to various data sources, consolidating metadata, and returning fact-based data points in response to user queries using LLMs and LangChain." 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "a310d6ea-2ee1-4979-bb5e-b65cb892c0cd", 22 | "metadata": {}, 23 | "source": [ 24 | "\n", 25 | "\n" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "id": "5d0297e0-f2dd-464b-9254-6693c45ebafc", 31 | "metadata": { 32 | "tags": [] 33 | }, 34 | "source": [ 35 | "Step 1. Connection to various channels through which LLMs can talk to your data. These channels include:\n", 36 | "\n", 37 | " - RedShift Serverless - to connect to datastore 'tickit'(ticket is referred as tickit in the sample data store) to retrieve information regarding ticket sales.\n", 38 | " - Aurora - MySQL Serverless - to connect to datastore that hosts information about the employees.\n", 39 | " - S3/Athena - to connect to the SageMaker's offline feature store on claims information. \n", 40 | " - Snowflake - to connect to stocks related data residing in finance schema of 3rd party software.\n", 41 | " - APIs - to connect to meteo(in this example we use Langchain's sample dataset on meteo) to retrieve weather information.\n", 42 | " \n", 43 | "Step 2. Usage of Dynamic generation of prompt templates by populating metadata of the tables using Glue Data Catalog(GDC) as context. GDC was populated by running a crawler on the databases. Refer to the information here to create and run a glue crawler. In case of api, a line item was created in GDC data extract.\n", 44 | "\n", 45 | "Step 3. Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query\n", 46 | "\n", 47 | "Step 4. Apply user query to LLM and Langchain to determine the data channel. After determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 48 | "\n", 49 | "Finally, display the results.\n" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "id": "e0986ea2-f794-431f-a341-b94f0118cb7d", 55 | "metadata": { 56 | "tags": [] 57 | }, 58 | "source": [ 59 | "### Pre-requisites:\n", 60 | "1. Use kernel Base Python 3.0.\n", 61 | "2. Deploy resources using the cloudformation template mda-llm-cfn.yml.\n", 62 | "\n", 63 | "[OPTIONAL] - If need is to add any of the sources below, then uncomment code in the relevant sections.\n", 64 | "\n", 65 | "1. Setup [Aurora MySQL Serverless database](https://aws.amazon.com/getting-started/hands-on/building-serverless-applications-with-amazon-aurora-serverless/?ref=gsrchandson). Load sample dataset for Human Resource department. Use this notebook to load the data into Aurora MySQL.\n", 66 | "2. Setup [Redshift Serverless](https://catalog.workshops.aws/redshift-immersion/en-US/lab1). Load sample data for Sales & Marketing. For example, 'sample data dev' for 'tickit' dataset available in RedShift examples.\n", 67 | "3. Setup External database. In this case, we are using Snowflake account and populating stocks data. Use this notebook to load the data into Snowflake.\n", 68 | "4. Add/modify the [Glue Crawler](https://catalog.us-east-1.prod.workshops.aws/workshops/71b5bdcf-7eb1-4549-b851-66adc860cd04/en-US/2-studio/1-crawler) on all the databases mentioned above. \n", 69 | "\n", 70 | "**Note - This notebook was tested on kernel - conda_python3 in Region us-east-1**" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "id": "9556eddc-8e45-4e42-9157-213316ec468a", 77 | "metadata": { 78 | "tags": [] 79 | }, 80 | "outputs": [], 81 | "source": [ 82 | "%%writefile requirements.txt\n", 83 | "sqlalchemy==1.4.47\n", 84 | "snowflake-sqlalchemy\n", 85 | "langchain==0.0.166\n", 86 | "sqlalchemy-aurora-data-api\n", 87 | "PyAthena[SQLAlchemy]==2.25.2\n", 88 | "redshift-connector==2.0.910\n", 89 | "sqlalchemy-redshift==0.8.14" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": null, 95 | "id": "b55d516c", 96 | "metadata": { 97 | "scrolled": true, 98 | "tags": [] 99 | }, 100 | "outputs": [], 101 | "source": [ 102 | "!pip install -r requirements.txt" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "id": "91c153cd", 109 | "metadata": { 110 | "tags": [] 111 | }, 112 | "outputs": [], 113 | "source": [ 114 | "import json\n", 115 | "import boto3\n", 116 | "\n", 117 | "import sqlalchemy\n", 118 | "from sqlalchemy import create_engine\n", 119 | "from snowflake.sqlalchemy import URL\n", 120 | "\n", 121 | "from langchain.docstore.document import Document\n", 122 | "from langchain import PromptTemplate,SagemakerEndpoint,SQLDatabase, SQLDatabaseChain, LLMChain\n", 123 | "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n", 124 | "from langchain.chains.question_answering import load_qa_chain\n", 125 | "from langchain.prompts.prompt import PromptTemplate\n", 126 | "from langchain.chains import SQLDatabaseSequentialChain\n", 127 | "\n", 128 | "from langchain.chains.api.prompt import API_RESPONSE_PROMPT\n", 129 | "from langchain.chains import APIChain\n", 130 | "from langchain.prompts.prompt import PromptTemplate\n", 131 | "\n", 132 | "from langchain.chains.api import open_meteo_docs\n", 133 | "\n", 134 | "from typing import Dict" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": null, 140 | "id": "f9c445a3-0f6b-4164-83f4-bd5ddf8d084c", 141 | "metadata": { 142 | "tags": [] 143 | }, 144 | "outputs": [], 145 | "source": [ 146 | "#define content handler class for flant5xl model\n", 147 | "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n", 148 | "class ContentHandler(LLMContentHandler):\n", 149 | " content_type = \"application/json\"\n", 150 | " accepts = \"application/json\"\n", 151 | " \n", 152 | " def transform_input(self, prompt, model_kwargs) :\n", 153 | " test = {\"text_inputs\": prompt}\n", 154 | " encoded_json = json.dumps(test).encode(\"utf-8\")\n", 155 | " return encoded_json\n", 156 | " \n", 157 | " def transform_output(self, output):\n", 158 | " response_json = json.loads(output.read().decode(\"utf-8\")).get('generated_texts')\n", 159 | " print(\"response\" , response_json)\n", 160 | " return \"\".join(response_json)\n", 161 | "\n", 162 | "content_handler = ContentHandler()" 163 | ] 164 | }, 165 | { 166 | "cell_type": "markdown", 167 | "id": "074a4144-4053-46c1-ba39-8f64f8fb9e00", 168 | "metadata": {}, 169 | "source": [ 170 | "The data for this COVID-19 dataset is stored in a public accessible S3 bucket. You can use the following command to explore the dataset.\n", 171 | "\n", 172 | "!aws s3 ls s3://covid19-lake/ --no-sign-request" 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "id": "87f05601-529a-42d4-8cab-ba0b9445e695", 178 | "metadata": { 179 | "tags": [] 180 | }, 181 | "source": [ 182 | "### Read parameters from Cloud Formation stack\n", 183 | "Some of the resources needed for this notebook such as the LLM model endpoint, the AWS Glue database and Glue crawler are created through a cloud formation template. The next block of code extracts the outputs and parameters of the cloud formation stack created from that template to get the value of these parameters.\n", 184 | "\n", 185 | "*The stack name here should match the stack name you used when creating the cloud formation stack.*" 186 | ] 187 | }, 188 | { 189 | "cell_type": "code", 190 | "execution_count": null, 191 | "id": "406e3ede-99aa-46e0-b3aa-eb8b65f6cff5", 192 | "metadata": { 193 | "tags": [] 194 | }, 195 | "outputs": [], 196 | "source": [ 197 | "# if used a different name while creating the cloud formation stack then change this to match the name you used\n", 198 | "CFN_STACK_NAME = \"cfn-genai-mda\"" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": null, 204 | "id": "86a1efa5-a1ae-4384-b149-4103b70fab56", 205 | "metadata": { 206 | "tags": [] 207 | }, 208 | "outputs": [], 209 | "source": [ 210 | "stacks = boto3.client('cloudformation').list_stacks()\n", 211 | "stack_found = CFN_STACK_NAME in [stack['StackName'] for stack in stacks['StackSummaries']]" 212 | ] 213 | }, 214 | { 215 | "cell_type": "code", 216 | "execution_count": null, 217 | "id": "c0bda7a4-f2a9-4379-b4a9-8e3e17240d0f", 218 | "metadata": { 219 | "tags": [] 220 | }, 221 | "outputs": [], 222 | "source": [ 223 | "from typing import List\n", 224 | "def get_cfn_outputs(stackname: str) -> List:\n", 225 | " cfn = boto3.client('cloudformation')\n", 226 | " outputs = {}\n", 227 | " for output in cfn.describe_stacks(StackName=stackname)['Stacks'][0]['Outputs']:\n", 228 | " outputs[output['OutputKey']] = output['OutputValue']\n", 229 | " return outputs\n", 230 | "\n", 231 | "def get_cfn_parameters(stackname: str) -> List:\n", 232 | " cfn = boto3.client('cloudformation')\n", 233 | " params = {}\n", 234 | " for param in cfn.describe_stacks(StackName=stackname)['Stacks'][0]['Parameters']:\n", 235 | " params[param['ParameterKey']] = param['ParameterValue']\n", 236 | " return params\n", 237 | "\n", 238 | "if stack_found is True:\n", 239 | " outputs = get_cfn_outputs(CFN_STACK_NAME)\n", 240 | " params = get_cfn_parameters(CFN_STACK_NAME)\n", 241 | " LLMEndpointName = outputs['LLMEndpointName']\n", 242 | " glue_crawler_name = params['CFNCrawlerName']\n", 243 | " glue_database_name = params['CFNDatabaseName']\n", 244 | " glue_databucket_name = params['DataBucketName']\n", 245 | " region = outputs['Region']\n", 246 | " print(f\"cfn outputs={outputs}\\nparams={params}\")\n", 247 | "else:\n", 248 | " print(\"Recheck our cloudformation stack name\")" 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "id": "59c5c0b3-d32e-41ff-85fe-dc80586fa737", 254 | "metadata": { 255 | "tags": [] 256 | }, 257 | "source": [ 258 | "### Copy the sample dataset to your S3 bucket" 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "execution_count": null, 264 | "id": "ceedd61d-9c21-45b4-b35e-69e5cd047f11", 265 | "metadata": { 266 | "tags": [] 267 | }, 268 | "outputs": [], 269 | "source": [ 270 | "!aws s3 cp --recursive s3://covid19-lake/rearc-covid-19-testing-data/json/states_daily/ s3://{glue_databucket_name}/covid-dataset/" 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "id": "7dbd2b73-cec1-4dbf-8a94-692eb04e7979", 276 | "metadata": { 277 | "tags": [] 278 | }, 279 | "source": [ 280 | "### Run the crawler" 281 | ] 282 | }, 283 | { 284 | "cell_type": "code", 285 | "execution_count": null, 286 | "id": "5ccbd062-0918-4dcf-a4c2-45bfa81e56c7", 287 | "metadata": { 288 | "tags": [] 289 | }, 290 | "outputs": [], 291 | "source": [ 292 | "%%writefile python_glueworkshop.py\n", 293 | "import boto3\n", 294 | "import argparse\n", 295 | "import time\n", 296 | "\n", 297 | "argParser = argparse.ArgumentParser()\n", 298 | "argParser.add_argument(\"-c\", \"--glue_crawler_name\", help=\"script help\")\n", 299 | "args = argParser.parse_args()\n", 300 | "print(args.glue_crawler_name )\n", 301 | "client = boto3.client('glue')\n", 302 | "crawler_name=args.glue_crawler_name\n", 303 | "\n", 304 | "def get_crawler_status(crawler_name):\n", 305 | " # Create a Glue client\n", 306 | " glue_client = boto3.client('glue')\n", 307 | "\n", 308 | " # Get the crawler details\n", 309 | " response = glue_client.get_crawler(Name=crawler_name)\n", 310 | "\n", 311 | " # Extract the crawler state\n", 312 | " crawler_state = response['Crawler']['State']\n", 313 | "\n", 314 | " return crawler_state\n", 315 | "\n", 316 | "# This is the command to start the Crawler\n", 317 | "try:\n", 318 | " response = client.start_crawler(Name=crawler_name )\n", 319 | " print(\"Successfully started crawler. The crawler may take 2-5 mins to detect the schema.\")\n", 320 | "\n", 321 | " while True:\n", 322 | " # Get the crawler status\n", 323 | " status = get_crawler_status(crawler_name)\n", 324 | "\n", 325 | " # Print the crawler status\n", 326 | " print(f\"Crawler '{crawler_name}' status: {status}\")\n", 327 | "\n", 328 | " if status == 'READY': # Replace 'READY' with the desired completed state\n", 329 | " break # Exit the loop if the desired state is reached\n", 330 | "\n", 331 | " time.sleep(10) # Sleep for 10 seconds before checking the status again\n", 332 | " \n", 333 | "except:\n", 334 | " print(\"error in starting crawler. Check the logs for the error details.\")\n" 335 | ] 336 | }, 337 | { 338 | "cell_type": "markdown", 339 | "id": "d1392725-610f-43c1-8fd9-b5f52c4585b1", 340 | "metadata": {}, 341 | "source": [ 342 | "Execute the python script by passing the glue crawler name from the cloudformation stack output." 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": null, 348 | "id": "75d0fa47-1651-4cff-802a-22ae6438a09c", 349 | "metadata": { 350 | "tags": [] 351 | }, 352 | "outputs": [], 353 | "source": [ 354 | "!python python_glueworkshop.py -c {glue_crawler_name}" 355 | ] 356 | }, 357 | { 358 | "cell_type": "markdown", 359 | "id": "4132ffc3-6947-49b6-b627-fae3df870b88", 360 | "metadata": { 361 | "tags": [] 362 | }, 363 | "source": [ 364 | "Before proceeding to the next step, check the status of the crawler. It should change from RUNNING to READY. " 365 | ] 366 | }, 367 | { 368 | "cell_type": "markdown", 369 | "id": "b51d1d0e-33fb-46ca-b82f-6294ea867cae", 370 | "metadata": { 371 | "tags": [] 372 | }, 373 | "source": [ 374 | "### Step 1 - Connect to databases using SQL Alchemy. \n", 375 | "\n", 376 | "Under the hood, LangChain uses SQLAlchemy to connect to SQL databases. The SQLDatabaseChain can therefore be used with any SQL dialect supported by SQLAlchemy, \n", 377 | "such as MS SQL, MySQL, MariaDB, PostgreSQL, Oracle SQL, and SQLite. Please refer to the SQLAlchemy documentation for more information about requirements for connecting to your database. \n" 378 | ] 379 | }, 380 | { 381 | "cell_type": "markdown", 382 | "id": "e5f5ce28-9b33-4061-8655-2b297d5c24a2", 383 | "metadata": { 384 | "tags": [] 385 | }, 386 | "source": [ 387 | "**Important**: The code below establishes a database connection for data sources and Large Language Models. Please note that the solution will only work if the database connection for your sources is defined in the cell below. Please refer to the Pre-requisites section. If your use case requires data from Aurora MySQL alone, then please comment out other data sources. Furthermore, please update the cluster details and variables for Aurora MySQL accordingly." 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "execution_count": null, 393 | "id": "1583cade", 394 | "metadata": { 395 | "scrolled": true, 396 | "tags": [] 397 | }, 398 | "outputs": [], 399 | "source": [ 400 | "#define connections\n", 401 | "\n", 402 | "#LLM \n", 403 | "llm=SagemakerEndpoint(\n", 404 | " endpoint_name=LLMEndpointName, \n", 405 | " region_name=region,\n", 406 | " model_kwargs={\"temperature\":1e-10},\n", 407 | " content_handler=content_handler\n", 408 | " )\n", 409 | "\n", 410 | "#S3\n", 411 | "# connect to s3 using athena\n", 412 | "## athena variables\n", 413 | "connathena=f\"athena.{region}.amazonaws.com\" \n", 414 | "portathena='443' #Update, if port is different\n", 415 | "schemaathena=glue_database_name #from cfn params\n", 416 | "s3stagingathena=f's3://{glue_databucket_name}/athenaresults/'#from cfn params\n", 417 | "wkgrpathena='primary'#Update, if workgroup is different\n", 418 | "# tablesathena=['dataset']#[]\n", 419 | "## Create the athena connection string\n", 420 | "connection_string = f\"awsathena+rest://@{connathena}:{portathena}/{schemaathena}?s3_staging_dir={s3stagingathena}/&work_group={wkgrpathena}\"\n", 421 | "## Create the athena SQLAlchemy engine\n", 422 | "engine_athena = create_engine(connection_string, echo=False)\n", 423 | "dbathena = SQLDatabase(engine_athena)\n", 424 | "# dbathena = SQLDatabase(engine_athena, include_tables=tablesathena)\n", 425 | "\n", 426 | "# collect credentials from Secrets Manager\n", 427 | "#Refer here on how to use AWS Secrets Manager - https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html\n", 428 | "# client = boto3.client('secretsmanager')\n", 429 | "\n", 430 | "# #SNOWFLAKE\n", 431 | "# # connect to snowflake database\n", 432 | "# ## snowflake variables\n", 433 | "# sf_account_id = \n", 434 | "# sf_secret_id =\n", 435 | "# dwh = \n", 436 | "# db = \n", 437 | "# schema = \n", 438 | "# table = \n", 439 | "# ## snowflake get credentials from secrets manager\n", 440 | "# response = client.get_secret_value(SecretId=sf_secret_id)\n", 441 | "# secrets_credentials = json.loads(response['SecretString'])\n", 442 | "# sf_password = secrets_credentials['password']\n", 443 | "# sf_username = secrets_credentials['username']\n", 444 | "# ## Create the snowflake connection string\n", 445 | "# connection_string = f\"snowflake://{sf_username}:{sf_password}@{sf_account_id}/{db}/{schema}?warehouse={dwh}\"\n", 446 | "# ## Create the snowflake SQLAlchemy engine\n", 447 | "# engine_snowflake = create_engine(connection_string, echo=False)\n", 448 | "# dbsnowflake = SQLDatabase(engine_snowflake)\n", 449 | "\n", 450 | "# #AURORA MYSQL\n", 451 | "# ##connect to aurora mysql\n", 452 | "# ##aurora mysql cluster details/variables\n", 453 | "# cluster_arn = \n", 454 | "# secret_arn =\n", 455 | "# rdsdb=\n", 456 | "# rdsdb_tbl = [
]\n", 457 | "# ## Create the aurora connection string\n", 458 | "# connection_string = f\"mysql+auroradataapi://:@/{rdsdb}\"\n", 459 | "# ## Create the aurora SQLAlchemy engine\n", 460 | "# engine_rds = create_engine(connection_string, echo=False,connect_args=dict(aurora_cluster_arn=cluster_arn, secret_arn=secret_arn))\n", 461 | "# dbrds = SQLDatabase(engine_rds, include_tables=rdsdb_tbl)\n", 462 | "\n", 463 | "# #REDSHIFT\n", 464 | "# # connect to redshift database\n", 465 | "# ## redshift variables\n", 466 | "# rs_secret_id = \n", 467 | "# rs_endpoint=\n", 468 | "# rs_port=\n", 469 | "# rs_db=\n", 470 | "# rs_schema=\n", 471 | "# ## redshift get credentials from secrets manager\n", 472 | "# response = client.get_secret_value(SecretId=rs_secret_id)\n", 473 | "# secrets_credentials = json.loads(response['SecretString'])\n", 474 | "# rs_password = secrets_credentials['password']\n", 475 | "# rs_username = secrets_credentials['username']\n", 476 | "# ## Create the redshift connection string\n", 477 | "# connection_string = f\"redshift+redshift_connector://{rs_username}:{rs_password}@{rs_endpoint}:{rs_port}/{rs_db}\"\n", 478 | "# engine_redshift = create_engine(connection_string, echo=False)\n", 479 | "# dbredshift = SQLDatabase(engine_redshift)\n", 480 | "\n", 481 | "#Glue Data Catalog\n", 482 | "##Provide list of all the databases where the table metadata resides after the glue successfully crawls the table\n", 483 | "# gdc = ['redshift-sagemaker-sample-data-dev', 'snowflake','rds-aurora-mysql-employees','sagemaker_featurestore'] # mentioned a few examples here\n", 484 | "gdc = [schemaathena] \n" 485 | ] 486 | }, 487 | { 488 | "cell_type": "markdown", 489 | "id": "1ea21757-b08a-438b-a5a7-79d85a9a9085", 490 | "metadata": {}, 491 | "source": [ 492 | "### Step 2 - Generate Dynamic Prompt Templates\n", 493 | "Build a consolidated view of Glue Data Catalog by combining metadata stored for all the databases in pipe delimited format." 494 | ] 495 | }, 496 | { 497 | "cell_type": "code", 498 | "execution_count": 12, 499 | "id": "08a3373d-9285-4fab-81b5-51e5364590b5", 500 | "metadata": { 501 | "scrolled": true, 502 | "tags": [] 503 | }, 504 | "outputs": [ 505 | { 506 | "name": "stdout", 507 | "output_type": "stream", 508 | "text": [ 509 | "s3|cfn_covid_lake|cfn_covid_dataset|totaltestresults\n", 510 | "s3|cfn_covid_lake|cfn_covid_dataset|fips\n", 511 | "s3|cfn_covid_lake|cfn_covid_dataset|deathincrease\n", 512 | "s3|cfn_covid_lake|cfn_covid_dataset|hospitalizedincrease\n", 513 | "s3|cfn_covid_lake|cfn_covid_dataset|negativeincrease\n", 514 | "s3|cfn_covid_lake|cfn_covid_dataset|positiveincrease\n", 515 | "s3|cfn_covid_lake|cfn_covid_dataset|totaltestresultsincrease\n", 516 | "s3|cfn_covid_lake|cfn_covid_dataset|negative\n", 517 | "s3|cfn_covid_lake|cfn_covid_dataset|pending\n", 518 | "api|meteo|weather|weather\n" 519 | ] 520 | } 521 | ], 522 | "source": [ 523 | "#Generate Dynamic prompts to populate the Glue Data Catalog\n", 524 | "#harvest aws crawler metadata\n", 525 | "\n", 526 | "def parse_catalog():\n", 527 | " #Connect to Glue catalog\n", 528 | " #get metadata of redshift serverless tables\n", 529 | " columns_str=''\n", 530 | " \n", 531 | " #define glue cient\n", 532 | " glue_client = boto3.client('glue')\n", 533 | " \n", 534 | " for db in gdc:\n", 535 | " response = glue_client.get_tables(DatabaseName =db)\n", 536 | " for tables in response['TableList']:\n", 537 | " #classification in the response for s3 and other databases is different. Set classification based on the response location\n", 538 | " if tables['StorageDescriptor']['Location'].startswith('s3'): classification='s3' \n", 539 | " else: classification = tables['Parameters']['classification']\n", 540 | " for columns in tables['StorageDescriptor']['Columns']:\n", 541 | " dbname,tblname,colname=tables['DatabaseName'],tables['Name'],columns['Name']\n", 542 | " columns_str=columns_str+f'\\n{classification}|{dbname}|{tblname}|{colname}' \n", 543 | " #API\n", 544 | " ## Append the metadata of the API to the unified glue data catalog\n", 545 | " columns_str=columns_str+'\\n'+('api|meteo|weather|weather')\n", 546 | " return columns_str\n", 547 | "\n", 548 | "glue_catalog = parse_catalog()\n", 549 | "\n", 550 | "#display a few lines from the catalog\n", 551 | "print('\\n'.join(glue_catalog.splitlines()[-10:]) )\n" 552 | ] 553 | }, 554 | { 555 | "cell_type": "markdown", 556 | "id": "a94e6770-42c3-402b-a60e-9c21fb99d5f6", 557 | "metadata": { 558 | "tags": [] 559 | }, 560 | "source": [ 561 | "### Step 3 - Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query" 562 | ] 563 | }, 564 | { 565 | "cell_type": "markdown", 566 | "id": "adda3714-3f32-4480-9526-91cca37489d1", 567 | "metadata": {}, 568 | "source": [ 569 | "In this code sample, we use the Anthropic Model to generate inferences. You can utilize SageMaker JumpStart models to achieve the same. \n", 570 | "Guidance on how to use the JumpStart Models is available in the notebook - mda_with_llm_langchain_smjumpstart_flant5xl" 571 | ] 572 | }, 573 | { 574 | "cell_type": "code", 575 | "execution_count": null, 576 | "id": "4efcc59b", 577 | "metadata": { 578 | "tags": [] 579 | }, 580 | "outputs": [], 581 | "source": [ 582 | "#Function 1 'Infer Channel'\n", 583 | "#define a function that infers the channel/database/table and sets the database for querying\n", 584 | "def identify_channel(query):\n", 585 | " #Prompt 1 'Infer Channel'\n", 586 | " ##set prompt template. It instructs the llm on how to evaluate and respond to the llm. It is referred to as dynamic since glue data catalog is first getting generated and appended to the prompt.\n", 587 | " prompt_template = \"\"\"\n", 588 | " From the table below, find the database (in column database) which will contain the data (in corresponding column_names) to answer the question \n", 589 | " {query} \\n\n", 590 | " \"\"\"+glue_catalog +\"\"\" \n", 591 | " Give your answer as database == \n", 592 | " Also,give your answer as database.table == \n", 593 | " \"\"\"\n", 594 | " ##define prompt 1\n", 595 | " PROMPT_channel = PromptTemplate( template=prompt_template, input_variables=[\"query\"] )\n", 596 | "\n", 597 | " # define llm chain\n", 598 | " llm_chain = LLMChain(prompt=PROMPT_channel, llm=llm)\n", 599 | " #run the query and save to generated texts\n", 600 | " generated_texts = llm_chain.run(query)\n", 601 | " print(generated_texts)\n", 602 | "\n", 603 | " #set the best channel from where the query can be answered\n", 604 | " if 'snowflake' in generated_texts: \n", 605 | " channel='db'\n", 606 | " db=dbsnowflake \n", 607 | " print(\"SET database to snowflake\") \n", 608 | " elif 'redshift' in generated_texts: \n", 609 | " channel='db'\n", 610 | " db=dbredshift\n", 611 | " print(\"SET database to redshift\")\n", 612 | " elif 's3' in generated_texts: \n", 613 | " channel='db'\n", 614 | " db=dbathena\n", 615 | " print(\"SET database to athena\")\n", 616 | " elif 'rdsmysql' in generated_texts: \n", 617 | " channel='db'\n", 618 | " db=dbrds\n", 619 | " print(\"SET database to rds\") \n", 620 | " elif 'api' in generated_texts: \n", 621 | " channel='api'\n", 622 | " print(\"SET database to weather api\") \n", 623 | " else: raise Exception(\"User question cannot be answered by any of the channels mentioned in the catalog\")\n", 624 | " print(\"Step complete. Channel is: \", channel)\n", 625 | " \n", 626 | " return channel, db\n", 627 | "\n", 628 | "#Function 2 'Run Query'\n", 629 | "#define a function that infers the channel/database/table and sets the database for querying\n", 630 | "def run_query(query):\n", 631 | "\n", 632 | " channel, db = identify_channel(query) #call the identify channel function first\n", 633 | "\n", 634 | " ##Prompt 2 'Run Query'\n", 635 | " #after determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 636 | " #provide rules for running the SQL queries in default template--> table info.\n", 637 | "\n", 638 | " _DEFAULT_TEMPLATE = \"\"\"Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.\n", 639 | "\n", 640 | " Only use the following tables:\n", 641 | "\n", 642 | " {table_info}\n", 643 | " if someone asks for covid data, then use the table cfn_covid_lake.cfn_covid_dataset.\n", 644 | "\n", 645 | " Question: {input}\"\"\"\n", 646 | "\n", 647 | " PROMPT_sql = PromptTemplate(\n", 648 | " input_variables=[\"input\", \"table_info\", \"dialect\"], template=_DEFAULT_TEMPLATE\n", 649 | " )\n", 650 | "\n", 651 | " \n", 652 | " if channel=='db':\n", 653 | " db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT_sql, verbose=False, return_intermediate_steps=False)\n", 654 | " response=db_chain.run(query)\n", 655 | " elif channel=='api':\n", 656 | " chain_api = APIChain.from_llm_and_api_docs(llm, open_meteo_docs.OPEN_METEO_DOCS, verbose=True)\n", 657 | " response=chain_api.run(query)\n", 658 | " else: raise Exception(\"Unlisted channel. Check your unified catalog\")\n", 659 | " return response\n", 660 | "\n" 661 | ] 662 | }, 663 | { 664 | "cell_type": "markdown", 665 | "id": "390a92cd-e1b4-4feb-ab7a-f97030ba7f84", 666 | "metadata": {}, 667 | "source": [ 668 | "### Step 4 - Run the run_query function that in turn calls the Langchain SQL Database chain to convert 'text to sql' and runs the query against the source data channel\n", 669 | "\n", 670 | "Some samples are provided below for test runs. Uncomment the query to run." 671 | ] 672 | }, 673 | { 674 | "cell_type": "code", 675 | "execution_count": 14, 676 | "id": "f82599a2", 677 | "metadata": { 678 | "tags": [] 679 | }, 680 | "outputs": [ 681 | { 682 | "name": "stdout", 683 | "output_type": "stream", 684 | "text": [ 685 | "response ['s3|cfn_covid_lake|cfn_covid']\n", 686 | "s3|cfn_covid_lake|cfn_covid\n", 687 | "SET database to athena\n", 688 | "Step complete. Channel is: db\n", 689 | "response ['SELECT count(*) FROM cfn_covid_dataset W']\n", 690 | "response ['[103900]']\n", 691 | "----------------------------------------------------------------------\n", 692 | "SQL and response from user query How many covid cases are there? \n", 693 | " [103900]\n" 694 | ] 695 | } 696 | ], 697 | "source": [ 698 | "# Enter the query\n", 699 | "## Few queries to try out - \n", 700 | "#athena - Healthcare - Covid dataset\n", 701 | "# query = \"\"\"How many covid hospitalizations were reported in NY in June of 2021?\"\"\" \n", 702 | "# query = \"\"\"Which States reported the least and maximum deaths?\"\"\" \n", 703 | "query=\"How many covid cases are there?\"\n", 704 | "\n", 705 | "#snowflake - Finance and Investments\n", 706 | "# query = \"\"\"Which stock performed the best and the worst in May of 2013?\"\"\"\n", 707 | "# query = \"\"\"What is the average volume stocks traded in July of 2013?\"\"\"\n", 708 | "\n", 709 | "#rds - Human Resources\n", 710 | "# query = \"\"\"Name all employees with birth date this month\"\"\" \n", 711 | "# query = \"\"\"Combien d'employés sont des femmes? \"\"\" #Ask question in French - How many females are there?\n", 712 | "# query = \"\"\"How many employees were hired before 1990?\"\"\" \n", 713 | "\n", 714 | "#athena - Legal - SageMaker offline featurestore\n", 715 | "# query = \"\"\"How many frauds happened in the year 2023 ?\"\"\" \n", 716 | "# query = \"\"\"How many policies were claimed this year ?\"\"\" \n", 717 | "\n", 718 | "#redshift - Sales & Marketing\n", 719 | "# query = \"\"\"How many tickit sales are there\"\"\" \n", 720 | "# query = \"what was the total commision for the tickit sales in the year 2008?\" \n", 721 | "\n", 722 | "#api - product - weather\n", 723 | "# query = \"\"\"What is the weather like right now in New York City in degrees Farenheit?\"\"\"\n", 724 | "\n", 725 | "#Response from Langchain\n", 726 | "response = run_query(query)\n", 727 | "print(\"----------------------------------------------------------------------\")\n", 728 | "print(f'SQL and response from user query {query} \\n {response}')" 729 | ] 730 | }, 731 | { 732 | "cell_type": "markdown", 733 | "id": "69371bdc-537f-4e5e-a004-99d852097862", 734 | "metadata": {}, 735 | "source": [ 736 | "### Clean-up\n", 737 | "After you run the modern data architecture with Generative AI, make sure to clean up any resources that won’t be utilized. Shutdown and delete the databases used (Amazon Redshift, Amazon RDS, Snowflake). In addition, delete the data in Amazon S3 and make sure to stop any SageMaker Studio notebook instances to not incur any further charges. If you used SageMaker Jumpstart to deploy large language model as SageMaker Real-time Endpoint, delete endpoint either through SageMaker console, or through Studio. \n", 738 | "\n", 739 | "To completely remove all the provisoned resources, go to CloudFormation and delete the stack.\n" 740 | ] 741 | } 742 | ], 743 | "metadata": { 744 | "availableInstances": [ 745 | { 746 | "_defaultOrder": 0, 747 | "_isFastLaunch": true, 748 | "category": "General purpose", 749 | "gpuNum": 0, 750 | "hideHardwareSpecs": false, 751 | "memoryGiB": 4, 752 | "name": "ml.t3.medium", 753 | "vcpuNum": 2 754 | }, 755 | { 756 | "_defaultOrder": 1, 757 | "_isFastLaunch": false, 758 | "category": "General purpose", 759 | "gpuNum": 0, 760 | "hideHardwareSpecs": false, 761 | "memoryGiB": 8, 762 | "name": "ml.t3.large", 763 | "vcpuNum": 2 764 | }, 765 | { 766 | "_defaultOrder": 2, 767 | "_isFastLaunch": false, 768 | "category": "General purpose", 769 | "gpuNum": 0, 770 | "hideHardwareSpecs": false, 771 | "memoryGiB": 16, 772 | "name": "ml.t3.xlarge", 773 | "vcpuNum": 4 774 | }, 775 | { 776 | "_defaultOrder": 3, 777 | "_isFastLaunch": false, 778 | "category": "General purpose", 779 | "gpuNum": 0, 780 | "hideHardwareSpecs": false, 781 | "memoryGiB": 32, 782 | "name": "ml.t3.2xlarge", 783 | "vcpuNum": 8 784 | }, 785 | { 786 | "_defaultOrder": 4, 787 | "_isFastLaunch": true, 788 | "category": "General purpose", 789 | "gpuNum": 0, 790 | "hideHardwareSpecs": false, 791 | "memoryGiB": 8, 792 | "name": "ml.m5.large", 793 | "vcpuNum": 2 794 | }, 795 | { 796 | "_defaultOrder": 5, 797 | "_isFastLaunch": false, 798 | "category": "General purpose", 799 | "gpuNum": 0, 800 | "hideHardwareSpecs": false, 801 | "memoryGiB": 16, 802 | "name": "ml.m5.xlarge", 803 | "vcpuNum": 4 804 | }, 805 | { 806 | "_defaultOrder": 6, 807 | "_isFastLaunch": false, 808 | "category": "General purpose", 809 | "gpuNum": 0, 810 | "hideHardwareSpecs": false, 811 | "memoryGiB": 32, 812 | "name": "ml.m5.2xlarge", 813 | "vcpuNum": 8 814 | }, 815 | { 816 | "_defaultOrder": 7, 817 | "_isFastLaunch": false, 818 | "category": "General purpose", 819 | "gpuNum": 0, 820 | "hideHardwareSpecs": false, 821 | "memoryGiB": 64, 822 | "name": "ml.m5.4xlarge", 823 | "vcpuNum": 16 824 | }, 825 | { 826 | "_defaultOrder": 8, 827 | "_isFastLaunch": false, 828 | "category": "General purpose", 829 | "gpuNum": 0, 830 | "hideHardwareSpecs": false, 831 | "memoryGiB": 128, 832 | "name": "ml.m5.8xlarge", 833 | "vcpuNum": 32 834 | }, 835 | { 836 | "_defaultOrder": 9, 837 | "_isFastLaunch": false, 838 | "category": "General purpose", 839 | "gpuNum": 0, 840 | "hideHardwareSpecs": false, 841 | "memoryGiB": 192, 842 | "name": "ml.m5.12xlarge", 843 | "vcpuNum": 48 844 | }, 845 | { 846 | "_defaultOrder": 10, 847 | "_isFastLaunch": false, 848 | "category": "General purpose", 849 | "gpuNum": 0, 850 | "hideHardwareSpecs": false, 851 | "memoryGiB": 256, 852 | "name": "ml.m5.16xlarge", 853 | "vcpuNum": 64 854 | }, 855 | { 856 | "_defaultOrder": 11, 857 | "_isFastLaunch": false, 858 | "category": "General purpose", 859 | "gpuNum": 0, 860 | "hideHardwareSpecs": false, 861 | "memoryGiB": 384, 862 | "name": "ml.m5.24xlarge", 863 | "vcpuNum": 96 864 | }, 865 | { 866 | "_defaultOrder": 12, 867 | "_isFastLaunch": false, 868 | "category": "General purpose", 869 | "gpuNum": 0, 870 | "hideHardwareSpecs": false, 871 | "memoryGiB": 8, 872 | "name": "ml.m5d.large", 873 | "vcpuNum": 2 874 | }, 875 | { 876 | "_defaultOrder": 13, 877 | "_isFastLaunch": false, 878 | "category": "General purpose", 879 | "gpuNum": 0, 880 | "hideHardwareSpecs": false, 881 | "memoryGiB": 16, 882 | "name": "ml.m5d.xlarge", 883 | "vcpuNum": 4 884 | }, 885 | { 886 | "_defaultOrder": 14, 887 | "_isFastLaunch": false, 888 | "category": "General purpose", 889 | "gpuNum": 0, 890 | "hideHardwareSpecs": false, 891 | "memoryGiB": 32, 892 | "name": "ml.m5d.2xlarge", 893 | "vcpuNum": 8 894 | }, 895 | { 896 | "_defaultOrder": 15, 897 | "_isFastLaunch": false, 898 | "category": "General purpose", 899 | "gpuNum": 0, 900 | "hideHardwareSpecs": false, 901 | "memoryGiB": 64, 902 | "name": "ml.m5d.4xlarge", 903 | "vcpuNum": 16 904 | }, 905 | { 906 | "_defaultOrder": 16, 907 | "_isFastLaunch": false, 908 | "category": "General purpose", 909 | "gpuNum": 0, 910 | "hideHardwareSpecs": false, 911 | "memoryGiB": 128, 912 | "name": "ml.m5d.8xlarge", 913 | "vcpuNum": 32 914 | }, 915 | { 916 | "_defaultOrder": 17, 917 | "_isFastLaunch": false, 918 | "category": "General purpose", 919 | "gpuNum": 0, 920 | "hideHardwareSpecs": false, 921 | "memoryGiB": 192, 922 | "name": "ml.m5d.12xlarge", 923 | "vcpuNum": 48 924 | }, 925 | { 926 | "_defaultOrder": 18, 927 | "_isFastLaunch": false, 928 | "category": "General purpose", 929 | "gpuNum": 0, 930 | "hideHardwareSpecs": false, 931 | "memoryGiB": 256, 932 | "name": "ml.m5d.16xlarge", 933 | "vcpuNum": 64 934 | }, 935 | { 936 | "_defaultOrder": 19, 937 | "_isFastLaunch": false, 938 | "category": "General purpose", 939 | "gpuNum": 0, 940 | "hideHardwareSpecs": false, 941 | "memoryGiB": 384, 942 | "name": "ml.m5d.24xlarge", 943 | "vcpuNum": 96 944 | }, 945 | { 946 | "_defaultOrder": 20, 947 | "_isFastLaunch": false, 948 | "category": "General purpose", 949 | "gpuNum": 0, 950 | "hideHardwareSpecs": true, 951 | "memoryGiB": 0, 952 | "name": "ml.geospatial.interactive", 953 | "supportedImageNames": [ 954 | "sagemaker-geospatial-v1-0" 955 | ], 956 | "vcpuNum": 0 957 | }, 958 | { 959 | "_defaultOrder": 21, 960 | "_isFastLaunch": true, 961 | "category": "Compute optimized", 962 | "gpuNum": 0, 963 | "hideHardwareSpecs": false, 964 | "memoryGiB": 4, 965 | "name": "ml.c5.large", 966 | "vcpuNum": 2 967 | }, 968 | { 969 | "_defaultOrder": 22, 970 | "_isFastLaunch": false, 971 | "category": "Compute optimized", 972 | "gpuNum": 0, 973 | "hideHardwareSpecs": false, 974 | "memoryGiB": 8, 975 | "name": "ml.c5.xlarge", 976 | "vcpuNum": 4 977 | }, 978 | { 979 | "_defaultOrder": 23, 980 | "_isFastLaunch": false, 981 | "category": "Compute optimized", 982 | "gpuNum": 0, 983 | "hideHardwareSpecs": false, 984 | "memoryGiB": 16, 985 | "name": "ml.c5.2xlarge", 986 | "vcpuNum": 8 987 | }, 988 | { 989 | "_defaultOrder": 24, 990 | "_isFastLaunch": false, 991 | "category": "Compute optimized", 992 | "gpuNum": 0, 993 | "hideHardwareSpecs": false, 994 | "memoryGiB": 32, 995 | "name": "ml.c5.4xlarge", 996 | "vcpuNum": 16 997 | }, 998 | { 999 | "_defaultOrder": 25, 1000 | "_isFastLaunch": false, 1001 | "category": "Compute optimized", 1002 | "gpuNum": 0, 1003 | "hideHardwareSpecs": false, 1004 | "memoryGiB": 72, 1005 | "name": "ml.c5.9xlarge", 1006 | "vcpuNum": 36 1007 | }, 1008 | { 1009 | "_defaultOrder": 26, 1010 | "_isFastLaunch": false, 1011 | "category": "Compute optimized", 1012 | "gpuNum": 0, 1013 | "hideHardwareSpecs": false, 1014 | "memoryGiB": 96, 1015 | "name": "ml.c5.12xlarge", 1016 | "vcpuNum": 48 1017 | }, 1018 | { 1019 | "_defaultOrder": 27, 1020 | "_isFastLaunch": false, 1021 | "category": "Compute optimized", 1022 | "gpuNum": 0, 1023 | "hideHardwareSpecs": false, 1024 | "memoryGiB": 144, 1025 | "name": "ml.c5.18xlarge", 1026 | "vcpuNum": 72 1027 | }, 1028 | { 1029 | "_defaultOrder": 28, 1030 | "_isFastLaunch": false, 1031 | "category": "Compute optimized", 1032 | "gpuNum": 0, 1033 | "hideHardwareSpecs": false, 1034 | "memoryGiB": 192, 1035 | "name": "ml.c5.24xlarge", 1036 | "vcpuNum": 96 1037 | }, 1038 | { 1039 | "_defaultOrder": 29, 1040 | "_isFastLaunch": true, 1041 | "category": "Accelerated computing", 1042 | "gpuNum": 1, 1043 | "hideHardwareSpecs": false, 1044 | "memoryGiB": 16, 1045 | "name": "ml.g4dn.xlarge", 1046 | "vcpuNum": 4 1047 | }, 1048 | { 1049 | "_defaultOrder": 30, 1050 | "_isFastLaunch": false, 1051 | "category": "Accelerated computing", 1052 | "gpuNum": 1, 1053 | "hideHardwareSpecs": false, 1054 | "memoryGiB": 32, 1055 | "name": "ml.g4dn.2xlarge", 1056 | "vcpuNum": 8 1057 | }, 1058 | { 1059 | "_defaultOrder": 31, 1060 | "_isFastLaunch": false, 1061 | "category": "Accelerated computing", 1062 | "gpuNum": 1, 1063 | "hideHardwareSpecs": false, 1064 | "memoryGiB": 64, 1065 | "name": "ml.g4dn.4xlarge", 1066 | "vcpuNum": 16 1067 | }, 1068 | { 1069 | "_defaultOrder": 32, 1070 | "_isFastLaunch": false, 1071 | "category": "Accelerated computing", 1072 | "gpuNum": 1, 1073 | "hideHardwareSpecs": false, 1074 | "memoryGiB": 128, 1075 | "name": "ml.g4dn.8xlarge", 1076 | "vcpuNum": 32 1077 | }, 1078 | { 1079 | "_defaultOrder": 33, 1080 | "_isFastLaunch": false, 1081 | "category": "Accelerated computing", 1082 | "gpuNum": 4, 1083 | "hideHardwareSpecs": false, 1084 | "memoryGiB": 192, 1085 | "name": "ml.g4dn.12xlarge", 1086 | "vcpuNum": 48 1087 | }, 1088 | { 1089 | "_defaultOrder": 34, 1090 | "_isFastLaunch": false, 1091 | "category": "Accelerated computing", 1092 | "gpuNum": 1, 1093 | "hideHardwareSpecs": false, 1094 | "memoryGiB": 256, 1095 | "name": "ml.g4dn.16xlarge", 1096 | "vcpuNum": 64 1097 | }, 1098 | { 1099 | "_defaultOrder": 35, 1100 | "_isFastLaunch": false, 1101 | "category": "Accelerated computing", 1102 | "gpuNum": 1, 1103 | "hideHardwareSpecs": false, 1104 | "memoryGiB": 61, 1105 | "name": "ml.p3.2xlarge", 1106 | "vcpuNum": 8 1107 | }, 1108 | { 1109 | "_defaultOrder": 36, 1110 | "_isFastLaunch": false, 1111 | "category": "Accelerated computing", 1112 | "gpuNum": 4, 1113 | "hideHardwareSpecs": false, 1114 | "memoryGiB": 244, 1115 | "name": "ml.p3.8xlarge", 1116 | "vcpuNum": 32 1117 | }, 1118 | { 1119 | "_defaultOrder": 37, 1120 | "_isFastLaunch": false, 1121 | "category": "Accelerated computing", 1122 | "gpuNum": 8, 1123 | "hideHardwareSpecs": false, 1124 | "memoryGiB": 488, 1125 | "name": "ml.p3.16xlarge", 1126 | "vcpuNum": 64 1127 | }, 1128 | { 1129 | "_defaultOrder": 38, 1130 | "_isFastLaunch": false, 1131 | "category": "Accelerated computing", 1132 | "gpuNum": 8, 1133 | "hideHardwareSpecs": false, 1134 | "memoryGiB": 768, 1135 | "name": "ml.p3dn.24xlarge", 1136 | "vcpuNum": 96 1137 | }, 1138 | { 1139 | "_defaultOrder": 39, 1140 | "_isFastLaunch": false, 1141 | "category": "Memory Optimized", 1142 | "gpuNum": 0, 1143 | "hideHardwareSpecs": false, 1144 | "memoryGiB": 16, 1145 | "name": "ml.r5.large", 1146 | "vcpuNum": 2 1147 | }, 1148 | { 1149 | "_defaultOrder": 40, 1150 | "_isFastLaunch": false, 1151 | "category": "Memory Optimized", 1152 | "gpuNum": 0, 1153 | "hideHardwareSpecs": false, 1154 | "memoryGiB": 32, 1155 | "name": "ml.r5.xlarge", 1156 | "vcpuNum": 4 1157 | }, 1158 | { 1159 | "_defaultOrder": 41, 1160 | "_isFastLaunch": false, 1161 | "category": "Memory Optimized", 1162 | "gpuNum": 0, 1163 | "hideHardwareSpecs": false, 1164 | "memoryGiB": 64, 1165 | "name": "ml.r5.2xlarge", 1166 | "vcpuNum": 8 1167 | }, 1168 | { 1169 | "_defaultOrder": 42, 1170 | "_isFastLaunch": false, 1171 | "category": "Memory Optimized", 1172 | "gpuNum": 0, 1173 | "hideHardwareSpecs": false, 1174 | "memoryGiB": 128, 1175 | "name": "ml.r5.4xlarge", 1176 | "vcpuNum": 16 1177 | }, 1178 | { 1179 | "_defaultOrder": 43, 1180 | "_isFastLaunch": false, 1181 | "category": "Memory Optimized", 1182 | "gpuNum": 0, 1183 | "hideHardwareSpecs": false, 1184 | "memoryGiB": 256, 1185 | "name": "ml.r5.8xlarge", 1186 | "vcpuNum": 32 1187 | }, 1188 | { 1189 | "_defaultOrder": 44, 1190 | "_isFastLaunch": false, 1191 | "category": "Memory Optimized", 1192 | "gpuNum": 0, 1193 | "hideHardwareSpecs": false, 1194 | "memoryGiB": 384, 1195 | "name": "ml.r5.12xlarge", 1196 | "vcpuNum": 48 1197 | }, 1198 | { 1199 | "_defaultOrder": 45, 1200 | "_isFastLaunch": false, 1201 | "category": "Memory Optimized", 1202 | "gpuNum": 0, 1203 | "hideHardwareSpecs": false, 1204 | "memoryGiB": 512, 1205 | "name": "ml.r5.16xlarge", 1206 | "vcpuNum": 64 1207 | }, 1208 | { 1209 | "_defaultOrder": 46, 1210 | "_isFastLaunch": false, 1211 | "category": "Memory Optimized", 1212 | "gpuNum": 0, 1213 | "hideHardwareSpecs": false, 1214 | "memoryGiB": 768, 1215 | "name": "ml.r5.24xlarge", 1216 | "vcpuNum": 96 1217 | }, 1218 | { 1219 | "_defaultOrder": 47, 1220 | "_isFastLaunch": false, 1221 | "category": "Accelerated computing", 1222 | "gpuNum": 1, 1223 | "hideHardwareSpecs": false, 1224 | "memoryGiB": 16, 1225 | "name": "ml.g5.xlarge", 1226 | "vcpuNum": 4 1227 | }, 1228 | { 1229 | "_defaultOrder": 48, 1230 | "_isFastLaunch": false, 1231 | "category": "Accelerated computing", 1232 | "gpuNum": 1, 1233 | "hideHardwareSpecs": false, 1234 | "memoryGiB": 32, 1235 | "name": "ml.g5.2xlarge", 1236 | "vcpuNum": 8 1237 | }, 1238 | { 1239 | "_defaultOrder": 49, 1240 | "_isFastLaunch": false, 1241 | "category": "Accelerated computing", 1242 | "gpuNum": 1, 1243 | "hideHardwareSpecs": false, 1244 | "memoryGiB": 64, 1245 | "name": "ml.g5.4xlarge", 1246 | "vcpuNum": 16 1247 | }, 1248 | { 1249 | "_defaultOrder": 50, 1250 | "_isFastLaunch": false, 1251 | "category": "Accelerated computing", 1252 | "gpuNum": 1, 1253 | "hideHardwareSpecs": false, 1254 | "memoryGiB": 128, 1255 | "name": "ml.g5.8xlarge", 1256 | "vcpuNum": 32 1257 | }, 1258 | { 1259 | "_defaultOrder": 51, 1260 | "_isFastLaunch": false, 1261 | "category": "Accelerated computing", 1262 | "gpuNum": 1, 1263 | "hideHardwareSpecs": false, 1264 | "memoryGiB": 256, 1265 | "name": "ml.g5.16xlarge", 1266 | "vcpuNum": 64 1267 | }, 1268 | { 1269 | "_defaultOrder": 52, 1270 | "_isFastLaunch": false, 1271 | "category": "Accelerated computing", 1272 | "gpuNum": 4, 1273 | "hideHardwareSpecs": false, 1274 | "memoryGiB": 192, 1275 | "name": "ml.g5.12xlarge", 1276 | "vcpuNum": 48 1277 | }, 1278 | { 1279 | "_defaultOrder": 53, 1280 | "_isFastLaunch": false, 1281 | "category": "Accelerated computing", 1282 | "gpuNum": 4, 1283 | "hideHardwareSpecs": false, 1284 | "memoryGiB": 384, 1285 | "name": "ml.g5.24xlarge", 1286 | "vcpuNum": 96 1287 | }, 1288 | { 1289 | "_defaultOrder": 54, 1290 | "_isFastLaunch": false, 1291 | "category": "Accelerated computing", 1292 | "gpuNum": 8, 1293 | "hideHardwareSpecs": false, 1294 | "memoryGiB": 768, 1295 | "name": "ml.g5.48xlarge", 1296 | "vcpuNum": 192 1297 | } 1298 | ], 1299 | "instance_type": "ml.t3.medium", 1300 | "kernelspec": { 1301 | "display_name": "conda_python3", 1302 | "language": "python", 1303 | "name": "conda_python3" 1304 | }, 1305 | "language_info": { 1306 | "codemirror_mode": { 1307 | "name": "ipython", 1308 | "version": 3 1309 | }, 1310 | "file_extension": ".py", 1311 | "mimetype": "text/x-python", 1312 | "name": "python", 1313 | "nbconvert_exporter": "python", 1314 | "pygments_lexer": "ipython3", 1315 | "version": "3.10.8" 1316 | } 1317 | }, 1318 | "nbformat": 4, 1319 | "nbformat_minor": 5 1320 | } 1321 | -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/mda_with_llm_langchain_byo_model_without_cloudformation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0855b561", 6 | "metadata": {}, 7 | "source": [ 8 | "# Integrate Modern Data Architectures with Generative AI and interact using prompts for querying SQL databases & APIs" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "26105729-b3e3-42d0-a583-8446fff89277", 14 | "metadata": {}, 15 | "source": [ 16 | "This notebook demonstrates how **_large language models, such as Anthropic,_** interact with AWS databases, data stores, and third-party data warehousing solutions like Snowflake. We showcase this interaction 1) by generating and running SQL queries, and 2) making requests to API endpoints. We achieve all of this by using the LangChain framework, which allows the language model to interact with its environment and connect with other sources of data. The LangChain framework operates based on the following principles: calling out to a language model, being data-aware, and being agentic. Our notebook focuses on establishing database connections to various data sources, consolidating metadata, and returning fact-based data points in response to user queries using LLMs and LangChain." 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "d02c8cc5-5104-44aa-bbce-ad3ca7562a29", 22 | "metadata": { 23 | "tags": [] 24 | }, 25 | "source": [ 26 | "This notebook focuses on establishing connection to one data source, consolidating metadata, and returning fact-based data points in response to user queries using LLMs and LangChain. The solution can be enhanced to add multiple data sources." 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "id": "33dc1232-e1fa-4e71-99c6-e72fc6cc66e4", 32 | "metadata": { 33 | "tags": [] 34 | }, 35 | "source": [ 36 | "
\n", 37 | "\n", 39 | "
Figure 1: Architecture
\n", 40 | "
" 41 | ] 42 | }, 43 | { 44 | "cell_type": "markdown", 45 | "id": "e0986ea2-f794-431f-a341-b94f0118cb7d", 46 | "metadata": { 47 | "tags": [] 48 | }, 49 | "source": [ 50 | "### Pre-requisites:\n", 51 | "1. Use kernel Base Python 3.0.\n", 52 | "2. Install the required packages.\n", 53 | "3. Run the One time Setup by entering the user input parameters, copying the dataset, setup IAM role and finally run the crawler.\n", 54 | "3. Access to the LLM API. In this notebook, Anthropic Model is used. Refer [here](https://console.anthropic.com/docs/access) for detais on how to get access to Anthropic API key.\n", 55 | "\n", 56 | "**Note - This notebook was tested on kernel - conda_python3 in Region us-east-1**" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "id": "9597c6f9", 62 | "metadata": {}, 63 | "source": [ 64 | "1. Attach AmazonAthenaFullAccess, AWSGlueServiceRole in IAM.\n", 65 | "2. Add the following custom policy in IAM to grant creating policy (double click cell to get json format)." 66 | ] 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "id": "4f9731fb", 71 | "metadata": {}, 72 | "source": [ 73 | "{\n", 74 | " \"Version\": \"2012-10-17\",\n", 75 | " \"Statement\": [\n", 76 | " {\n", 77 | " \"Action\": [\n", 78 | " \"iam:AttachRolePolicy\",\n", 79 | " \"iam:CreateRole\",\n", 80 | " \"iam:CreatePolicy\",\n", 81 | " \"iam:GetRole\",\n", 82 | " \"iam:PassRole\"\n", 83 | " ],\n", 84 | " \"Effect\": \"Allow\",\n", 85 | " \"Resource\": \"*\"\n", 86 | " }\n", 87 | " ]\n", 88 | "}" 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "id": "5d0297e0-f2dd-464b-9254-6693c45ebafc", 94 | "metadata": { 95 | "tags": [] 96 | }, 97 | "source": [ 98 | "### Solution Walkthrough:\n", 99 | "\n", 100 | "Step 1. Connection to S3 through which LLMs can talk to your data. These channels include:\n", 101 | " - S3/Athena - to connect to the SageMaker's offline feature store on claims information. \n", 102 | " \n", 103 | "Step 2. Usage of Dynamic generation of prompt templates by populating metadata of the tables using Glue Data Catalog(GDC) as context. GDC was populated by running a crawler on the databases. Refer to the information here to create and run a glue crawler. In case of api, a line item was created in GDC data extract.\n", 104 | "\n", 105 | "Step 3. Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query\n", 106 | "\n", 107 | "Step 4. Apply user query to LLM and Langchain to determine the data channel. After determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 108 | "\n", 109 | "Finally, display the results.\n" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "id": "9556eddc-8e45-4e42-9157-213316ec468a", 116 | "metadata": { 117 | "tags": [] 118 | }, 119 | "outputs": [], 120 | "source": [ 121 | "%%writefile requirements.txt\n", 122 | "sqlalchemy==1.4.47\n", 123 | "snowflake-sqlalchemy\n", 124 | "#langchain==0.0.166\n", 125 | "langchain==0.0.190\n", 126 | "sqlalchemy-aurora-data-api\n", 127 | "PyAthena[SQLAlchemy]==2.25.2\n", 128 | "anthropic\n", 129 | "redshift-connector==2.0.910\n", 130 | "sqlalchemy-redshift==0.8.14" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": null, 136 | "id": "b55d516c", 137 | "metadata": { 138 | "scrolled": true, 139 | "tags": [] 140 | }, 141 | "outputs": [], 142 | "source": [ 143 | "!pip install -r requirements.txt --quiet" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": null, 149 | "id": "91c153cd", 150 | "metadata": { 151 | "tags": [] 152 | }, 153 | "outputs": [], 154 | "source": [ 155 | "import json\n", 156 | "import boto3\n", 157 | "\n", 158 | "import sqlalchemy\n", 159 | "from sqlalchemy import create_engine\n", 160 | "from snowflake.sqlalchemy import URL\n", 161 | "\n", 162 | "from langchain.docstore.document import Document\n", 163 | "from langchain import PromptTemplate,SagemakerEndpoint,SQLDatabase, SQLDatabaseChain, LLMChain\n", 164 | "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n", 165 | "from langchain.chains.question_answering import load_qa_chain\n", 166 | "from langchain.prompts.prompt import PromptTemplate\n", 167 | "from langchain.chains import SQLDatabaseSequentialChain\n", 168 | "\n", 169 | "from langchain.chains.api.prompt import API_RESPONSE_PROMPT\n", 170 | "from langchain.chains import APIChain\n", 171 | "from langchain.prompts.prompt import PromptTemplate\n", 172 | "from langchain.chat_models import ChatAnthropic\n", 173 | "from langchain.chains.api import open_meteo_docs\n", 174 | "\n", 175 | "from typing import Dict\n", 176 | "import time" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "id": "074a4144-4053-46c1-ba39-8f64f8fb9e00", 182 | "metadata": {}, 183 | "source": [ 184 | "The data for this COVID-19 dataset is stored in a public accessible S3 bucket. You can use the following command to explore the dataset.\n", 185 | "\n", 186 | "!aws s3 ls s3://covid19-lake/ --no-sign-request" 187 | ] 188 | }, 189 | { 190 | "cell_type": "markdown", 191 | "id": "87f05601-529a-42d4-8cab-ba0b9445e695", 192 | "metadata": { 193 | "tags": [] 194 | }, 195 | "source": [ 196 | "### One Time Setup\n", 197 | "Some of the resources needed for this notebook such as the IAM policy, AWS Glue database and Glue crawler are created through a cloud formation template. The next block of code does the setup based on user inputs.\n", 198 | "\n", 199 | "**NOTE - The next two blocks of code need to be run only for the first time.**" 200 | ] 201 | }, 202 | { 203 | "cell_type": "markdown", 204 | "id": "fb560be7-cdea-4ff8-9230-0679252ecf5d", 205 | "metadata": { 206 | "tags": [] 207 | }, 208 | "source": [ 209 | "### User Input\n" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "id": "23ab506d-e40d-48ab-af71-82ebb78445c0", 216 | "metadata": { 217 | "tags": [] 218 | }, 219 | "outputs": [], 220 | "source": [ 221 | "#provide user input\n", 222 | "glue_databucket_name = 'blog-genai-mda' #Create this bucket in S3\n", 223 | "glue_db_name='genai-workshop200'\n", 224 | "glue_role= 'AWSGlueServiceRole-glueworkshop200'\n", 225 | "glue_crawler_name=glue_db_name+'-crawler200'" 226 | ] 227 | }, 228 | { 229 | "cell_type": "markdown", 230 | "id": "59c5c0b3-d32e-41ff-85fe-dc80586fa737", 231 | "metadata": { 232 | "tags": [] 233 | }, 234 | "source": [ 235 | "### Copy the sample dataset to **your S3 bucket**" 236 | ] 237 | }, 238 | { 239 | "cell_type": "code", 240 | "execution_count": null, 241 | "id": "ceedd61d-9c21-45b4-b35e-69e5cd047f11", 242 | "metadata": { 243 | "tags": [] 244 | }, 245 | "outputs": [], 246 | "source": [ 247 | "!aws s3 cp --recursive s3://covid19-lake/rearc-covid-19-testing-data/json/states_daily/ s3://{glue_databucket_name}/covid-dataset/" 248 | ] 249 | }, 250 | { 251 | "cell_type": "markdown", 252 | "id": "a6344f3f-40da-4ffb-8b4f-b27c52e1eb02", 253 | "metadata": { 254 | "tags": [] 255 | }, 256 | "source": [ 257 | "### Create IAM Role that runs the crawler" 258 | ] 259 | }, 260 | { 261 | "cell_type": "code", 262 | "execution_count": null, 263 | "id": "2d905b91", 264 | "metadata": { 265 | "tags": [] 266 | }, 267 | "outputs": [], 268 | "source": [ 269 | "import boto3\n", 270 | "import os\n", 271 | "# Retrieve the AWS account number\n", 272 | "sts_client = boto3.client('sts')\n", 273 | "account_number = sts_client.get_caller_identity().get('Account')\n", 274 | "# Retrieve the AWS region\n", 275 | "#region = os.environ['AWS_REGION']\n", 276 | "region = boto3.session.Session().region_name\n", 277 | "print(\"AWS Account Number:\", account_number)\n", 278 | "print(\"AWS Region:\", region)\n", 279 | "trust_policy=\"\"\"{\n", 280 | " \"Version\": \"2012-10-17\",\n", 281 | " \"Statement\": [\n", 282 | " {\n", 283 | " \"Sid\": \"\",\n", 284 | " \"Effect\": \"Allow\",\n", 285 | " \"Principal\": {\n", 286 | " \"Service\": \"glue.amazonaws.com\"\n", 287 | " },\n", 288 | " \"Action\": \"sts:AssumeRole\"\n", 289 | " }\n", 290 | " ]\n", 291 | "}\"\"\"\n", 292 | "managed_policy=\"\"\"{\n", 293 | " \"Version\": \"2012-10-17\",\n", 294 | " \"Statement\": [\n", 295 | " {\n", 296 | " \"Action\": [\n", 297 | " \"glue:*\"\n", 298 | " ],\n", 299 | " \"Resource\": [\n", 300 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":catalog\",\n", 301 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":database/*\",\n", 302 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":table/*\"\n", 303 | " ],\n", 304 | " \"Effect\": \"Allow\",\n", 305 | " \"Sid\": \"Readcrawlerresources\"\n", 306 | " },\n", 307 | " {\n", 308 | " \"Action\": [\n", 309 | " \"logs:CreateLogGroup\",\n", 310 | " \"logs:CreateLogStream\",\n", 311 | " \"logs:PutLogEvents\"\n", 312 | " ],\n", 313 | " \"Resource\": [\n", 314 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":log-group:/aws-glue/crawlers*\",\n", 315 | " \"arn:aws:logs:*:*:/aws-glue/*\",\n", 316 | " \"arn:aws:logs:*:*:/customlogs/*\"\n", 317 | " ],\n", 318 | " \"Effect\": \"Allow\",\n", 319 | " \"Sid\": \"ReadlogResources\"\n", 320 | " },\n", 321 | " {\n", 322 | " \"Action\": [\n", 323 | " \"s3:PutObject\",\n", 324 | " \"s3:GetObject\",\n", 325 | " \"s3:PutBucketLogging\",\n", 326 | " \"s3:ListBucket\",\n", 327 | " \"s3:PutBucketVersioning\"\n", 328 | " ],\n", 329 | " \"Resource\": [\n", 330 | " \"arn:aws:s3:::\"\"\"+glue_databucket_name+\"\"\"\",\n", 331 | " \"arn:aws:s3:::\"\"\"+glue_databucket_name+\"\"\"/*\"\n", 332 | " ],\n", 333 | " \"Effect\": \"Allow\",\n", 334 | " \"Sid\": \"ReadS3Resources\"\n", 335 | " }\n", 336 | " ]\n", 337 | " }\"\"\"\n", 338 | "print(managed_policy, file=open('managed-policy.json', 'w'))\n", 339 | "print(trust_policy, file=open('trust-policy.json', 'w'))" 340 | ] 341 | }, 342 | { 343 | "cell_type": "code", 344 | "execution_count": null, 345 | "id": "c4c27763-e1e4-4b41-bfa4-7ff8f6500c31", 346 | "metadata": { 347 | "tags": [] 348 | }, 349 | "outputs": [], 350 | "source": [ 351 | "%%sh -s \"$glue_role\" \n", 352 | "echo $1 \n", 353 | "glue_role=\"$1\"\n", 354 | "managed_policy_name=\"managed-policy-$glue_role\"\n", 355 | "echo $managed_policy_name\n", 356 | "aws iam create-role --role-name $glue_role --assume-role-policy-document file://trust-policy.json\n", 357 | "output=$(aws iam create-policy --policy-document file://managed-policy.json --policy-name $managed_policy_name)\n", 358 | "arn=$(echo \"$output\" | grep -oP '\"Arn\": \"\\K[^\"]+')\n", 359 | "echo \"$arn\"\n", 360 | "aws iam attach-role-policy --policy-arn $arn --role-name $glue_role" 361 | ] 362 | }, 363 | { 364 | "cell_type": "code", 365 | "execution_count": null, 366 | "id": "6fe28c2a-60c4-400a-9693-395e304c5164", 367 | "metadata": { 368 | "tags": [] 369 | }, 370 | "outputs": [], 371 | "source": [ 372 | "import boto3\n", 373 | "\n", 374 | "client = boto3.client('glue')\n", 375 | "\n", 376 | "# Create database \n", 377 | "try:\n", 378 | " response = client.create_database(\n", 379 | " DatabaseInput={\n", 380 | " 'Name': glue_db_name,\n", 381 | " 'Description': 'This database is created using Python boto3',\n", 382 | " }\n", 383 | " )\n", 384 | " print(\"Successfully created database\")\n", 385 | "except:\n", 386 | " print(\"error in creating database. Check if the database already exists\")\n", 387 | "\n", 388 | "#introducing some lag for the iam role to create\n", 389 | "time.sleep(20) \n", 390 | "\n", 391 | "# Create Glue Crawler \n", 392 | "try:\n", 393 | "\n", 394 | " response = client.create_crawler(\n", 395 | " Name=glue_crawler_name,\n", 396 | " Role=glue_role,\n", 397 | " DatabaseName=glue_db_name,\n", 398 | " Targets={\n", 399 | " 'S3Targets': [\n", 400 | " {\n", 401 | " 'Path': 's3://{BUCKET_NAME}/covid-dataset/'.format(BUCKET_NAME =glue_databucket_name)\n", 402 | " }\n", 403 | " ]\n", 404 | " },\n", 405 | " TablePrefix=''\n", 406 | " )\n", 407 | " \n", 408 | " print(\"Successfully created crawler\")\n", 409 | "except:\n", 410 | " print(\"error in creating crawler. However, if the crawler already exists, the crawler will run.\")\n", 411 | "\n", 412 | "# Run the Crawler\n", 413 | "try:\n", 414 | " response = client.start_crawler(Name=glue_crawler_name )\n", 415 | " print(\"Successfully started crawler. The crawler may take 2-5 mins to detect the schema.\")\n", 416 | " while True:\n", 417 | " # Get the crawler status\n", 418 | " response = client.get_crawler(Name=glue_crawler_name)\n", 419 | " # Extract the crawler state\n", 420 | " status = response['Crawler']['State']\n", 421 | " # Print the crawler status\n", 422 | " print(f\"Crawler '{glue_crawler_name}' status: {status}\")\n", 423 | " if status == 'READY': # Replace 'READY' with the desired completed state\n", 424 | " break # Exit the loop if the desired state is reached\n", 425 | "\n", 426 | " time.sleep(10) # Sleep for 10 seconds before checking the status again\n", 427 | " \n", 428 | "except:\n", 429 | " print(\"error in starting crawler. Check the logs for the error details.\")" 430 | ] 431 | }, 432 | { 433 | "cell_type": "markdown", 434 | "id": "4132ffc3-6947-49b6-b627-fae3df870b88", 435 | "metadata": { 436 | "tags": [] 437 | }, 438 | "source": [ 439 | "Before proceeding to the next step, check the status of the crawler. It should change from RUNNING to READY. " 440 | ] 441 | }, 442 | { 443 | "cell_type": "markdown", 444 | "id": "b51d1d0e-33fb-46ca-b82f-6294ea867cae", 445 | "metadata": { 446 | "tags": [] 447 | }, 448 | "source": [ 449 | "### Step 1 - Connect to databases using SQL Alchemy. \n", 450 | "\n", 451 | "Under the hood, LangChain uses SQLAlchemy to connect to SQL databases. The SQLDatabaseChain can therefore be used with any SQL dialect supported by SQLAlchemy, \n", 452 | "such as MS SQL, MySQL, MariaDB, PostgreSQL, Oracle SQL, and SQLite. Please refer to the SQLAlchemy documentation for more information about requirements for connecting to your database. \n" 453 | ] 454 | }, 455 | { 456 | "cell_type": "markdown", 457 | "id": "e5f5ce28-9b33-4061-8655-2b297d5c24a2", 458 | "metadata": { 459 | "tags": [] 460 | }, 461 | "source": [ 462 | "**Important**: The code below establishes a database connection for data sources and Large Language Models. Please note that the solution will only work if the database connection for your sources is defined in the cell below. Please refer to the Pre-requisites section. If your use case requires data from Aurora MySQL alone, then please comment out other data sources. Furthermore, please update the cluster details and variables for Aurora MySQL accordingly." 463 | ] 464 | }, 465 | { 466 | "cell_type": "code", 467 | "execution_count": null, 468 | "id": "1583cade", 469 | "metadata": { 470 | "scrolled": true, 471 | "tags": [] 472 | }, 473 | "outputs": [], 474 | "source": [ 475 | "#define connections\n", 476 | "\n", 477 | "# collect credentials from Secrets Manager\n", 478 | "#Refer here on how to use AWS Secrets Manager - https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html\n", 479 | "client = boto3.client('secretsmanager')\n", 480 | "region=client.meta.region_name\n", 481 | "\n", 482 | "#LLM \n", 483 | "#get the llm api key\n", 484 | "#llm variables\n", 485 | "#Refer here for access to Anthropic API Keys https://console.anthropic.com/docs/access\n", 486 | "anthropic_secret_id = \"anthropic\"#\n", 487 | "## llm get credentials from secrets manager\n", 488 | "response = client.get_secret_value(SecretId=anthropic_secret_id)\n", 489 | "secrets_credentials = json.loads(response['SecretString'])\n", 490 | "ANTHROPIC_API_KEY = secrets_credentials['ANTHROPIC_API_KEY']\n", 491 | "#define large language model here. Make sure to set api keys for the variable ANTHROPIC_API_KEY\n", 492 | "llm = ChatAnthropic(temperature=0, anthropic_api_key=ANTHROPIC_API_KEY, max_tokens_to_sample = 512)\n", 493 | "\n", 494 | "#S3\n", 495 | "# connect to s3 using athena\n", 496 | "## athena variables\n", 497 | "connathena=f\"athena.{region}.amazonaws.com\" \n", 498 | "portathena='443' #Update, if port is different\n", 499 | "schemaathena=glue_db_name #from user defined params\n", 500 | "s3stagingathena=f's3://{glue_databucket_name}/athenaresults/'#from cfn params\n", 501 | "wkgrpathena='primary'#Update, if workgroup is different\n", 502 | "# tablesathena=['dataset']#[]\n", 503 | "## Create the athena connection string\n", 504 | "connection_string = f\"awsathena+rest://@{connathena}:{portathena}/{schemaathena}?s3_staging_dir={s3stagingathena}/&work_group={wkgrpathena}\"\n", 505 | "## Create the athena SQLAlchemy engine\n", 506 | "engine_athena = create_engine(connection_string, echo=False)\n", 507 | "dbathena = SQLDatabase(engine_athena)\n", 508 | "\n", 509 | "gdc = [schemaathena] \n" 510 | ] 511 | }, 512 | { 513 | "cell_type": "markdown", 514 | "id": "1ea21757-b08a-438b-a5a7-79d85a9a9085", 515 | "metadata": {}, 516 | "source": [ 517 | "### Step 2 - Generate Dynamic Prompt Templates\n", 518 | "Build a consolidated view of Glue Data Catalog by combining metadata stored for all the databases in pipe delimited format." 519 | ] 520 | }, 521 | { 522 | "cell_type": "code", 523 | "execution_count": 11, 524 | "id": "08a3373d-9285-4fab-81b5-51e5364590b5", 525 | "metadata": { 526 | "scrolled": true, 527 | "tags": [] 528 | }, 529 | "outputs": [ 530 | { 531 | "name": "stdout", 532 | "output_type": "stream", 533 | "text": [ 534 | "s3|genai-workshop200|covid_dataset|totaltestresults\n", 535 | "s3|genai-workshop200|covid_dataset|fips\n", 536 | "s3|genai-workshop200|covid_dataset|deathincrease\n", 537 | "s3|genai-workshop200|covid_dataset|hospitalizedincrease\n", 538 | "s3|genai-workshop200|covid_dataset|negativeincrease\n", 539 | "s3|genai-workshop200|covid_dataset|positiveincrease\n", 540 | "s3|genai-workshop200|covid_dataset|totaltestresultsincrease\n", 541 | "s3|genai-workshop200|covid_dataset|negative\n", 542 | "s3|genai-workshop200|covid_dataset|pending\n", 543 | "api|meteo|weather|weather\n" 544 | ] 545 | } 546 | ], 547 | "source": [ 548 | "#Generate Dynamic prompts to populate the Glue Data Catalog\n", 549 | "#harvest aws crawler metadata\n", 550 | "\n", 551 | "def parse_catalog():\n", 552 | " #Connect to Glue catalog\n", 553 | " #get metadata of redshift serverless tables\n", 554 | " columns_str=''\n", 555 | " \n", 556 | " #define glue cient\n", 557 | " glue_client = boto3.client('glue')\n", 558 | " \n", 559 | " for db in gdc:\n", 560 | " response = glue_client.get_tables(DatabaseName =db)\n", 561 | " for tables in response['TableList']:\n", 562 | " #classification in the response for s3 and other databases is different. Set classification based on the response location\n", 563 | " if tables['StorageDescriptor']['Location'].startswith('s3'): classification='s3' \n", 564 | " else: classification = tables['Parameters']['classification']\n", 565 | " for columns in tables['StorageDescriptor']['Columns']:\n", 566 | " dbname,tblname,colname=tables['DatabaseName'],tables['Name'],columns['Name']\n", 567 | " columns_str=columns_str+f'\\n{classification}|{dbname}|{tblname}|{colname}' \n", 568 | " #API\n", 569 | " ## Append the metadata of the API to the unified glue data catalog\n", 570 | " columns_str=columns_str+'\\n'+('api|meteo|weather|weather')\n", 571 | " return columns_str\n", 572 | "\n", 573 | "glue_catalog = parse_catalog()\n", 574 | "\n", 575 | "#display a few lines from the catalog\n", 576 | "print('\\n'.join(glue_catalog.splitlines()[-10:]) )\n" 577 | ] 578 | }, 579 | { 580 | "cell_type": "markdown", 581 | "id": "a94e6770-42c3-402b-a60e-9c21fb99d5f6", 582 | "metadata": { 583 | "tags": [] 584 | }, 585 | "source": [ 586 | "### Step 3 - Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query" 587 | ] 588 | }, 589 | { 590 | "cell_type": "markdown", 591 | "id": "adda3714-3f32-4480-9526-91cca37489d1", 592 | "metadata": {}, 593 | "source": [ 594 | "In this code sample, we use the Anthropic Model to generate inferences. You can utilize SageMaker JumpStart models to achieve the same. \n", 595 | "Guidance on how to use the JumpStart Models is available in the notebook - mda_with_llm_langchain_smjumpstart_flant5xl" 596 | ] 597 | }, 598 | { 599 | "cell_type": "code", 600 | "execution_count": null, 601 | "id": "4efcc59b", 602 | "metadata": { 603 | "tags": [] 604 | }, 605 | "outputs": [], 606 | "source": [ 607 | "#Function 1 'Infer Channel'\n", 608 | "#define a function that infers the channel/database/table and sets the database for querying\n", 609 | "def identify_channel(query):\n", 610 | " #Prompt 1 'Infer Channel'\n", 611 | " ##set prompt template. It instructs the llm on how to evaluate and respond to the llm. It is referred to as dynamic since glue data catalog is first getting generated and appended to the prompt.\n", 612 | " prompt_template = \"\"\"\n", 613 | " From the table below, find the database (in column database) which will contain the data (in corresponding column_names) to answer the question \n", 614 | " {query} \\n\n", 615 | " \"\"\"+glue_catalog +\"\"\" \n", 616 | " Give your answer as database == \n", 617 | " Also,give your answer as database.table == \n", 618 | " \"\"\"\n", 619 | " ##define prompt 1\n", 620 | " PROMPT_channel = PromptTemplate( template=prompt_template, input_variables=[\"query\"] )\n", 621 | "\n", 622 | " # define llm chain\n", 623 | " llm_chain = LLMChain(prompt=PROMPT_channel, llm=llm)\n", 624 | " #run the query and save to generated texts\n", 625 | " generated_texts = llm_chain.run(query)\n", 626 | " print(generated_texts)\n", 627 | "\n", 628 | " #set the channel from where the query can be answered\n", 629 | " if 's3' in generated_texts: \n", 630 | " channel='db'\n", 631 | " db=dbathena\n", 632 | " print(\"SET database to athena\")\n", 633 | " elif 'api' in generated_texts: \n", 634 | " channel='api'\n", 635 | " print(\"SET database to weather api\") \n", 636 | " else: raise Exception(\"User question cannot be answered by any of the channels mentioned in the catalog\")\n", 637 | " print(\"Step complete. Channel is: \", channel)\n", 638 | " \n", 639 | " return channel, db\n", 640 | "\n", 641 | "#Function 2 'Run Query'\n", 642 | "#define a function that infers the channel/database/table and sets the database for querying\n", 643 | "def run_query(query):\n", 644 | "\n", 645 | " channel, db = identify_channel(query) #call the identify channel function first\n", 646 | "\n", 647 | " ##Prompt 2 'Run Query'\n", 648 | " #after determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 649 | " #provide rules for running the SQL queries in default template--> table info.\n", 650 | "\n", 651 | " _DEFAULT_TEMPLATE = \"\"\"Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.\n", 652 | "\n", 653 | " Do not append 'Query:' to SQLQuery.\n", 654 | " \n", 655 | " Display SQLResult after the query is run in plain english that users can understand. \n", 656 | "\n", 657 | " Provide answer in simple english statement.\n", 658 | " \n", 659 | " Only use the following tables:\n", 660 | "\n", 661 | " {table_info}\n", 662 | " If someone asks for the sales, they really mean the tickit.sales table.\n", 663 | " If someone asks for the sales date, they really mean the column tickit.sales.saletime.\n", 664 | "\n", 665 | " Question: {input}\"\"\"\n", 666 | "\n", 667 | " PROMPT_sql = PromptTemplate(\n", 668 | " input_variables=[\"input\", \"table_info\", \"dialect\"], template=_DEFAULT_TEMPLATE\n", 669 | " )\n", 670 | "\n", 671 | " \n", 672 | " if channel=='db':\n", 673 | " db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT_sql, verbose=True, return_intermediate_steps=False)\n", 674 | " response=db_chain.run(query)\n", 675 | " elif channel=='api':\n", 676 | " chain_api = APIChain.from_llm_and_api_docs(llm, open_meteo_docs.OPEN_METEO_DOCS, verbose=True)\n", 677 | " response=chain_api.run(query)\n", 678 | " else: raise Exception(\"Unlisted channel. Check your unified catalog\")\n", 679 | " return response\n", 680 | "\n" 681 | ] 682 | }, 683 | { 684 | "cell_type": "markdown", 685 | "id": "390a92cd-e1b4-4feb-ab7a-f97030ba7f84", 686 | "metadata": {}, 687 | "source": [ 688 | "### Step 4 - Run the run_query function that in turn calls the Langchain SQL Database chain to convert 'text to sql' and runs the query against the source data channel\n", 689 | "\n", 690 | "Some samples are provided below for test runs. Uncomment the query to run." 691 | ] 692 | }, 693 | { 694 | "cell_type": "code", 695 | "execution_count": 13, 696 | "id": "f82599a2", 697 | "metadata": { 698 | "tags": [] 699 | }, 700 | "outputs": [ 701 | { 702 | "name": "stdout", 703 | "output_type": "stream", 704 | "text": [ 705 | " database == s3|genai-workshop200|covid_dataset\n", 706 | "database.table == s3|genai-workshop200|covid_dataset\n", 707 | "SET database to athena\n", 708 | "Step complete. Channel is: db\n", 709 | "\n", 710 | "\n", 711 | "\u001b[1m> Entering new SQLDatabaseChain chain...\u001b[0m\n", 712 | "Which States reported the least and maximum deaths?\n", 713 | "SQLQuery:\u001b[32;1m\u001b[1;3mSELECT state, MIN(death) AS least_deaths, MAX(death) AS max_deaths \n", 714 | "FROM covid_dataset \n", 715 | "GROUP BY state \n", 716 | "ORDER BY least_deaths, max_deaths\u001b[0m\n", 717 | "SQLResult: \u001b[33;1m\u001b[1;3m[('AS', 0.0, 0.0), ('MP', 0.0, 2.0), ('AK', 0.0, 305.0), ('WY', 0.0, 682.0), ('DC', 0.0, 1030.0), ('DE', 0.0, 1473.0), ('ND', 0.0, 1478.0), ('ID', 0.0, 1879.0), ('UT', 0.0, 1976.0), ('NE', 0.0, 2113.0), ('WV', 0.0, 2325.0), ('NV', 0.0, 5037.0), ('AR', 0.0, 5417.0), ('MO', 0.0, 8161.0), ('AL', 0.0, 10149.0), ('NC', 0.0, 11502.0), ('IN', 0.0, 12737.0), ('AZ', 0.0, 16328.0), ('MI', 0.0, 16658.0), ('NJ', 0.0, 23574.0), ('VI', 1.0, 25.0), ('GU', 1.0, 133.0), ('HI', 1.0, 445.0), ('ME', 1.0, 706.0), ('NH', 1.0, 1184.0), ('MT', 1.0, 1381.0), ('SD', 1.0, 1900.0), ('PR', 1.0, 2059.0), ('NM', 1.0, 3808.0), ('OK', 1.0, 4534.0), ('KS', 1.0, 4816.0), ('KY', 1.0, 4819.0), ('IA', 1.0, 5558.0), ('CO', 1.0, 5989.0), ('MN', 1.0, 6550.0), ('MS', 1.0, 6808.0), ('CT', 1.0, 7704.0), ('SC', 1.0, 8754.0), ('VA', 1.0, 9596.0), ('OH', 1.0, 17656.0), ('GA', 1.0, 17906.0), ('IL', 1.0, 23014.0), ('PA', 1.0, 24349.0), ('TX', 1.0, 44451.0), ('VT', 2.0, 208.0), ('WA', 2.0, 5041.0), ('MD', 2.0, 7955.0), ('LA', 2.0, 9748.0), ('TN', 2.0, 11543.0), ('MA', 2.0, 16417.0), ('FL', 2.0, 32266.0), ('OR', 3.0, 2296.0), ('RI', 3.0, 2541.0), ('WI', 3.0, 7106.0), ('NY', 3.0, 39029.0), ('CA', 4.0, 54124.0)]\u001b[0m\n", 718 | "Answer:\u001b[32;1m\u001b[1;3mThe states that reported the least deaths are American Samoa, Northern Mariana Islands and Alaska.\n", 719 | "The states that reported the maximum deaths are California, New York and Florida.\u001b[0m\n", 720 | "\u001b[1m> Finished chain.\u001b[0m\n", 721 | "----------------------------------------------------------------------\n", 722 | "SQL and response from user query Which States reported the least and maximum deaths? \n", 723 | " The states that reported the least deaths are American Samoa, Northern Mariana Islands and Alaska.\n", 724 | "The states that reported the maximum deaths are California, New York and Florida.\n" 725 | ] 726 | } 727 | ], 728 | "source": [ 729 | "# Enter the query\n", 730 | "## Few queries to try out - \n", 731 | "#athena - Healthcare - Covid dataset\n", 732 | "# query = \"\"\"How many covid hospitalizations were reported in NY in June of 2021?\"\"\" \n", 733 | "query = \"\"\"Which States reported the least and maximum deaths?\"\"\" \n", 734 | "\n", 735 | "#api - product - weather\n", 736 | "# query = \"\"\"What is the weather like right now in New York City in degrees Farenheit?\"\"\"\n", 737 | "\n", 738 | "#Response from Langchain\n", 739 | "response = run_query(query)\n", 740 | "print(\"----------------------------------------------------------------------\")\n", 741 | "print(f'SQL and response from user query {query} \\n {response}')" 742 | ] 743 | }, 744 | { 745 | "cell_type": "markdown", 746 | "id": "69371bdc-537f-4e5e-a004-99d852097862", 747 | "metadata": {}, 748 | "source": [ 749 | "### Clean-up\n", 750 | "After you run the modern data architecture with Generative AI, make sure to clean up any resources that won’t be utilized. Delete the data in Amazon S3 and make sure to stop any SageMaker Studio notebook instances to not incur any further charges.\n" 751 | ] 752 | } 753 | ], 754 | "metadata": { 755 | "availableInstances": [ 756 | { 757 | "_defaultOrder": 0, 758 | "_isFastLaunch": true, 759 | "category": "General purpose", 760 | "gpuNum": 0, 761 | "hideHardwareSpecs": false, 762 | "memoryGiB": 4, 763 | "name": "ml.t3.medium", 764 | "vcpuNum": 2 765 | }, 766 | { 767 | "_defaultOrder": 1, 768 | "_isFastLaunch": false, 769 | "category": "General purpose", 770 | "gpuNum": 0, 771 | "hideHardwareSpecs": false, 772 | "memoryGiB": 8, 773 | "name": "ml.t3.large", 774 | "vcpuNum": 2 775 | }, 776 | { 777 | "_defaultOrder": 2, 778 | "_isFastLaunch": false, 779 | "category": "General purpose", 780 | "gpuNum": 0, 781 | "hideHardwareSpecs": false, 782 | "memoryGiB": 16, 783 | "name": "ml.t3.xlarge", 784 | "vcpuNum": 4 785 | }, 786 | { 787 | "_defaultOrder": 3, 788 | "_isFastLaunch": false, 789 | "category": "General purpose", 790 | "gpuNum": 0, 791 | "hideHardwareSpecs": false, 792 | "memoryGiB": 32, 793 | "name": "ml.t3.2xlarge", 794 | "vcpuNum": 8 795 | }, 796 | { 797 | "_defaultOrder": 4, 798 | "_isFastLaunch": true, 799 | "category": "General purpose", 800 | "gpuNum": 0, 801 | "hideHardwareSpecs": false, 802 | "memoryGiB": 8, 803 | "name": "ml.m5.large", 804 | "vcpuNum": 2 805 | }, 806 | { 807 | "_defaultOrder": 5, 808 | "_isFastLaunch": false, 809 | "category": "General purpose", 810 | "gpuNum": 0, 811 | "hideHardwareSpecs": false, 812 | "memoryGiB": 16, 813 | "name": "ml.m5.xlarge", 814 | "vcpuNum": 4 815 | }, 816 | { 817 | "_defaultOrder": 6, 818 | "_isFastLaunch": false, 819 | "category": "General purpose", 820 | "gpuNum": 0, 821 | "hideHardwareSpecs": false, 822 | "memoryGiB": 32, 823 | "name": "ml.m5.2xlarge", 824 | "vcpuNum": 8 825 | }, 826 | { 827 | "_defaultOrder": 7, 828 | "_isFastLaunch": false, 829 | "category": "General purpose", 830 | "gpuNum": 0, 831 | "hideHardwareSpecs": false, 832 | "memoryGiB": 64, 833 | "name": "ml.m5.4xlarge", 834 | "vcpuNum": 16 835 | }, 836 | { 837 | "_defaultOrder": 8, 838 | "_isFastLaunch": false, 839 | "category": "General purpose", 840 | "gpuNum": 0, 841 | "hideHardwareSpecs": false, 842 | "memoryGiB": 128, 843 | "name": "ml.m5.8xlarge", 844 | "vcpuNum": 32 845 | }, 846 | { 847 | "_defaultOrder": 9, 848 | "_isFastLaunch": false, 849 | "category": "General purpose", 850 | "gpuNum": 0, 851 | "hideHardwareSpecs": false, 852 | "memoryGiB": 192, 853 | "name": "ml.m5.12xlarge", 854 | "vcpuNum": 48 855 | }, 856 | { 857 | "_defaultOrder": 10, 858 | "_isFastLaunch": false, 859 | "category": "General purpose", 860 | "gpuNum": 0, 861 | "hideHardwareSpecs": false, 862 | "memoryGiB": 256, 863 | "name": "ml.m5.16xlarge", 864 | "vcpuNum": 64 865 | }, 866 | { 867 | "_defaultOrder": 11, 868 | "_isFastLaunch": false, 869 | "category": "General purpose", 870 | "gpuNum": 0, 871 | "hideHardwareSpecs": false, 872 | "memoryGiB": 384, 873 | "name": "ml.m5.24xlarge", 874 | "vcpuNum": 96 875 | }, 876 | { 877 | "_defaultOrder": 12, 878 | "_isFastLaunch": false, 879 | "category": "General purpose", 880 | "gpuNum": 0, 881 | "hideHardwareSpecs": false, 882 | "memoryGiB": 8, 883 | "name": "ml.m5d.large", 884 | "vcpuNum": 2 885 | }, 886 | { 887 | "_defaultOrder": 13, 888 | "_isFastLaunch": false, 889 | "category": "General purpose", 890 | "gpuNum": 0, 891 | "hideHardwareSpecs": false, 892 | "memoryGiB": 16, 893 | "name": "ml.m5d.xlarge", 894 | "vcpuNum": 4 895 | }, 896 | { 897 | "_defaultOrder": 14, 898 | "_isFastLaunch": false, 899 | "category": "General purpose", 900 | "gpuNum": 0, 901 | "hideHardwareSpecs": false, 902 | "memoryGiB": 32, 903 | "name": "ml.m5d.2xlarge", 904 | "vcpuNum": 8 905 | }, 906 | { 907 | "_defaultOrder": 15, 908 | "_isFastLaunch": false, 909 | "category": "General purpose", 910 | "gpuNum": 0, 911 | "hideHardwareSpecs": false, 912 | "memoryGiB": 64, 913 | "name": "ml.m5d.4xlarge", 914 | "vcpuNum": 16 915 | }, 916 | { 917 | "_defaultOrder": 16, 918 | "_isFastLaunch": false, 919 | "category": "General purpose", 920 | "gpuNum": 0, 921 | "hideHardwareSpecs": false, 922 | "memoryGiB": 128, 923 | "name": "ml.m5d.8xlarge", 924 | "vcpuNum": 32 925 | }, 926 | { 927 | "_defaultOrder": 17, 928 | "_isFastLaunch": false, 929 | "category": "General purpose", 930 | "gpuNum": 0, 931 | "hideHardwareSpecs": false, 932 | "memoryGiB": 192, 933 | "name": "ml.m5d.12xlarge", 934 | "vcpuNum": 48 935 | }, 936 | { 937 | "_defaultOrder": 18, 938 | "_isFastLaunch": false, 939 | "category": "General purpose", 940 | "gpuNum": 0, 941 | "hideHardwareSpecs": false, 942 | "memoryGiB": 256, 943 | "name": "ml.m5d.16xlarge", 944 | "vcpuNum": 64 945 | }, 946 | { 947 | "_defaultOrder": 19, 948 | "_isFastLaunch": false, 949 | "category": "General purpose", 950 | "gpuNum": 0, 951 | "hideHardwareSpecs": false, 952 | "memoryGiB": 384, 953 | "name": "ml.m5d.24xlarge", 954 | "vcpuNum": 96 955 | }, 956 | { 957 | "_defaultOrder": 20, 958 | "_isFastLaunch": false, 959 | "category": "General purpose", 960 | "gpuNum": 0, 961 | "hideHardwareSpecs": true, 962 | "memoryGiB": 0, 963 | "name": "ml.geospatial.interactive", 964 | "supportedImageNames": [ 965 | "sagemaker-geospatial-v1-0" 966 | ], 967 | "vcpuNum": 0 968 | }, 969 | { 970 | "_defaultOrder": 21, 971 | "_isFastLaunch": true, 972 | "category": "Compute optimized", 973 | "gpuNum": 0, 974 | "hideHardwareSpecs": false, 975 | "memoryGiB": 4, 976 | "name": "ml.c5.large", 977 | "vcpuNum": 2 978 | }, 979 | { 980 | "_defaultOrder": 22, 981 | "_isFastLaunch": false, 982 | "category": "Compute optimized", 983 | "gpuNum": 0, 984 | "hideHardwareSpecs": false, 985 | "memoryGiB": 8, 986 | "name": "ml.c5.xlarge", 987 | "vcpuNum": 4 988 | }, 989 | { 990 | "_defaultOrder": 23, 991 | "_isFastLaunch": false, 992 | "category": "Compute optimized", 993 | "gpuNum": 0, 994 | "hideHardwareSpecs": false, 995 | "memoryGiB": 16, 996 | "name": "ml.c5.2xlarge", 997 | "vcpuNum": 8 998 | }, 999 | { 1000 | "_defaultOrder": 24, 1001 | "_isFastLaunch": false, 1002 | "category": "Compute optimized", 1003 | "gpuNum": 0, 1004 | "hideHardwareSpecs": false, 1005 | "memoryGiB": 32, 1006 | "name": "ml.c5.4xlarge", 1007 | "vcpuNum": 16 1008 | }, 1009 | { 1010 | "_defaultOrder": 25, 1011 | "_isFastLaunch": false, 1012 | "category": "Compute optimized", 1013 | "gpuNum": 0, 1014 | "hideHardwareSpecs": false, 1015 | "memoryGiB": 72, 1016 | "name": "ml.c5.9xlarge", 1017 | "vcpuNum": 36 1018 | }, 1019 | { 1020 | "_defaultOrder": 26, 1021 | "_isFastLaunch": false, 1022 | "category": "Compute optimized", 1023 | "gpuNum": 0, 1024 | "hideHardwareSpecs": false, 1025 | "memoryGiB": 96, 1026 | "name": "ml.c5.12xlarge", 1027 | "vcpuNum": 48 1028 | }, 1029 | { 1030 | "_defaultOrder": 27, 1031 | "_isFastLaunch": false, 1032 | "category": "Compute optimized", 1033 | "gpuNum": 0, 1034 | "hideHardwareSpecs": false, 1035 | "memoryGiB": 144, 1036 | "name": "ml.c5.18xlarge", 1037 | "vcpuNum": 72 1038 | }, 1039 | { 1040 | "_defaultOrder": 28, 1041 | "_isFastLaunch": false, 1042 | "category": "Compute optimized", 1043 | "gpuNum": 0, 1044 | "hideHardwareSpecs": false, 1045 | "memoryGiB": 192, 1046 | "name": "ml.c5.24xlarge", 1047 | "vcpuNum": 96 1048 | }, 1049 | { 1050 | "_defaultOrder": 29, 1051 | "_isFastLaunch": true, 1052 | "category": "Accelerated computing", 1053 | "gpuNum": 1, 1054 | "hideHardwareSpecs": false, 1055 | "memoryGiB": 16, 1056 | "name": "ml.g4dn.xlarge", 1057 | "vcpuNum": 4 1058 | }, 1059 | { 1060 | "_defaultOrder": 30, 1061 | "_isFastLaunch": false, 1062 | "category": "Accelerated computing", 1063 | "gpuNum": 1, 1064 | "hideHardwareSpecs": false, 1065 | "memoryGiB": 32, 1066 | "name": "ml.g4dn.2xlarge", 1067 | "vcpuNum": 8 1068 | }, 1069 | { 1070 | "_defaultOrder": 31, 1071 | "_isFastLaunch": false, 1072 | "category": "Accelerated computing", 1073 | "gpuNum": 1, 1074 | "hideHardwareSpecs": false, 1075 | "memoryGiB": 64, 1076 | "name": "ml.g4dn.4xlarge", 1077 | "vcpuNum": 16 1078 | }, 1079 | { 1080 | "_defaultOrder": 32, 1081 | "_isFastLaunch": false, 1082 | "category": "Accelerated computing", 1083 | "gpuNum": 1, 1084 | "hideHardwareSpecs": false, 1085 | "memoryGiB": 128, 1086 | "name": "ml.g4dn.8xlarge", 1087 | "vcpuNum": 32 1088 | }, 1089 | { 1090 | "_defaultOrder": 33, 1091 | "_isFastLaunch": false, 1092 | "category": "Accelerated computing", 1093 | "gpuNum": 4, 1094 | "hideHardwareSpecs": false, 1095 | "memoryGiB": 192, 1096 | "name": "ml.g4dn.12xlarge", 1097 | "vcpuNum": 48 1098 | }, 1099 | { 1100 | "_defaultOrder": 34, 1101 | "_isFastLaunch": false, 1102 | "category": "Accelerated computing", 1103 | "gpuNum": 1, 1104 | "hideHardwareSpecs": false, 1105 | "memoryGiB": 256, 1106 | "name": "ml.g4dn.16xlarge", 1107 | "vcpuNum": 64 1108 | }, 1109 | { 1110 | "_defaultOrder": 35, 1111 | "_isFastLaunch": false, 1112 | "category": "Accelerated computing", 1113 | "gpuNum": 1, 1114 | "hideHardwareSpecs": false, 1115 | "memoryGiB": 61, 1116 | "name": "ml.p3.2xlarge", 1117 | "vcpuNum": 8 1118 | }, 1119 | { 1120 | "_defaultOrder": 36, 1121 | "_isFastLaunch": false, 1122 | "category": "Accelerated computing", 1123 | "gpuNum": 4, 1124 | "hideHardwareSpecs": false, 1125 | "memoryGiB": 244, 1126 | "name": "ml.p3.8xlarge", 1127 | "vcpuNum": 32 1128 | }, 1129 | { 1130 | "_defaultOrder": 37, 1131 | "_isFastLaunch": false, 1132 | "category": "Accelerated computing", 1133 | "gpuNum": 8, 1134 | "hideHardwareSpecs": false, 1135 | "memoryGiB": 488, 1136 | "name": "ml.p3.16xlarge", 1137 | "vcpuNum": 64 1138 | }, 1139 | { 1140 | "_defaultOrder": 38, 1141 | "_isFastLaunch": false, 1142 | "category": "Accelerated computing", 1143 | "gpuNum": 8, 1144 | "hideHardwareSpecs": false, 1145 | "memoryGiB": 768, 1146 | "name": "ml.p3dn.24xlarge", 1147 | "vcpuNum": 96 1148 | }, 1149 | { 1150 | "_defaultOrder": 39, 1151 | "_isFastLaunch": false, 1152 | "category": "Memory Optimized", 1153 | "gpuNum": 0, 1154 | "hideHardwareSpecs": false, 1155 | "memoryGiB": 16, 1156 | "name": "ml.r5.large", 1157 | "vcpuNum": 2 1158 | }, 1159 | { 1160 | "_defaultOrder": 40, 1161 | "_isFastLaunch": false, 1162 | "category": "Memory Optimized", 1163 | "gpuNum": 0, 1164 | "hideHardwareSpecs": false, 1165 | "memoryGiB": 32, 1166 | "name": "ml.r5.xlarge", 1167 | "vcpuNum": 4 1168 | }, 1169 | { 1170 | "_defaultOrder": 41, 1171 | "_isFastLaunch": false, 1172 | "category": "Memory Optimized", 1173 | "gpuNum": 0, 1174 | "hideHardwareSpecs": false, 1175 | "memoryGiB": 64, 1176 | "name": "ml.r5.2xlarge", 1177 | "vcpuNum": 8 1178 | }, 1179 | { 1180 | "_defaultOrder": 42, 1181 | "_isFastLaunch": false, 1182 | "category": "Memory Optimized", 1183 | "gpuNum": 0, 1184 | "hideHardwareSpecs": false, 1185 | "memoryGiB": 128, 1186 | "name": "ml.r5.4xlarge", 1187 | "vcpuNum": 16 1188 | }, 1189 | { 1190 | "_defaultOrder": 43, 1191 | "_isFastLaunch": false, 1192 | "category": "Memory Optimized", 1193 | "gpuNum": 0, 1194 | "hideHardwareSpecs": false, 1195 | "memoryGiB": 256, 1196 | "name": "ml.r5.8xlarge", 1197 | "vcpuNum": 32 1198 | }, 1199 | { 1200 | "_defaultOrder": 44, 1201 | "_isFastLaunch": false, 1202 | "category": "Memory Optimized", 1203 | "gpuNum": 0, 1204 | "hideHardwareSpecs": false, 1205 | "memoryGiB": 384, 1206 | "name": "ml.r5.12xlarge", 1207 | "vcpuNum": 48 1208 | }, 1209 | { 1210 | "_defaultOrder": 45, 1211 | "_isFastLaunch": false, 1212 | "category": "Memory Optimized", 1213 | "gpuNum": 0, 1214 | "hideHardwareSpecs": false, 1215 | "memoryGiB": 512, 1216 | "name": "ml.r5.16xlarge", 1217 | "vcpuNum": 64 1218 | }, 1219 | { 1220 | "_defaultOrder": 46, 1221 | "_isFastLaunch": false, 1222 | "category": "Memory Optimized", 1223 | "gpuNum": 0, 1224 | "hideHardwareSpecs": false, 1225 | "memoryGiB": 768, 1226 | "name": "ml.r5.24xlarge", 1227 | "vcpuNum": 96 1228 | }, 1229 | { 1230 | "_defaultOrder": 47, 1231 | "_isFastLaunch": false, 1232 | "category": "Accelerated computing", 1233 | "gpuNum": 1, 1234 | "hideHardwareSpecs": false, 1235 | "memoryGiB": 16, 1236 | "name": "ml.g5.xlarge", 1237 | "vcpuNum": 4 1238 | }, 1239 | { 1240 | "_defaultOrder": 48, 1241 | "_isFastLaunch": false, 1242 | "category": "Accelerated computing", 1243 | "gpuNum": 1, 1244 | "hideHardwareSpecs": false, 1245 | "memoryGiB": 32, 1246 | "name": "ml.g5.2xlarge", 1247 | "vcpuNum": 8 1248 | }, 1249 | { 1250 | "_defaultOrder": 49, 1251 | "_isFastLaunch": false, 1252 | "category": "Accelerated computing", 1253 | "gpuNum": 1, 1254 | "hideHardwareSpecs": false, 1255 | "memoryGiB": 64, 1256 | "name": "ml.g5.4xlarge", 1257 | "vcpuNum": 16 1258 | }, 1259 | { 1260 | "_defaultOrder": 50, 1261 | "_isFastLaunch": false, 1262 | "category": "Accelerated computing", 1263 | "gpuNum": 1, 1264 | "hideHardwareSpecs": false, 1265 | "memoryGiB": 128, 1266 | "name": "ml.g5.8xlarge", 1267 | "vcpuNum": 32 1268 | }, 1269 | { 1270 | "_defaultOrder": 51, 1271 | "_isFastLaunch": false, 1272 | "category": "Accelerated computing", 1273 | "gpuNum": 1, 1274 | "hideHardwareSpecs": false, 1275 | "memoryGiB": 256, 1276 | "name": "ml.g5.16xlarge", 1277 | "vcpuNum": 64 1278 | }, 1279 | { 1280 | "_defaultOrder": 52, 1281 | "_isFastLaunch": false, 1282 | "category": "Accelerated computing", 1283 | "gpuNum": 4, 1284 | "hideHardwareSpecs": false, 1285 | "memoryGiB": 192, 1286 | "name": "ml.g5.12xlarge", 1287 | "vcpuNum": 48 1288 | }, 1289 | { 1290 | "_defaultOrder": 53, 1291 | "_isFastLaunch": false, 1292 | "category": "Accelerated computing", 1293 | "gpuNum": 4, 1294 | "hideHardwareSpecs": false, 1295 | "memoryGiB": 384, 1296 | "name": "ml.g5.24xlarge", 1297 | "vcpuNum": 96 1298 | }, 1299 | { 1300 | "_defaultOrder": 54, 1301 | "_isFastLaunch": false, 1302 | "category": "Accelerated computing", 1303 | "gpuNum": 8, 1304 | "hideHardwareSpecs": false, 1305 | "memoryGiB": 768, 1306 | "name": "ml.g5.48xlarge", 1307 | "vcpuNum": 192 1308 | }, 1309 | { 1310 | "_defaultOrder": 55, 1311 | "_isFastLaunch": false, 1312 | "category": "Accelerated computing", 1313 | "gpuNum": 8, 1314 | "hideHardwareSpecs": false, 1315 | "memoryGiB": 1152, 1316 | "name": "ml.p4d.24xlarge", 1317 | "vcpuNum": 96 1318 | }, 1319 | { 1320 | "_defaultOrder": 56, 1321 | "_isFastLaunch": false, 1322 | "category": "Accelerated computing", 1323 | "gpuNum": 8, 1324 | "hideHardwareSpecs": false, 1325 | "memoryGiB": 1152, 1326 | "name": "ml.p4de.24xlarge", 1327 | "vcpuNum": 96 1328 | } 1329 | ], 1330 | "instance_type": "ml.t3.medium", 1331 | "kernelspec": { 1332 | "display_name": "conda_python3", 1333 | "language": "python", 1334 | "name": "conda_python3" 1335 | }, 1336 | "language_info": { 1337 | "codemirror_mode": { 1338 | "name": "ipython", 1339 | "version": 3 1340 | }, 1341 | "file_extension": ".py", 1342 | "mimetype": "text/x-python", 1343 | "name": "python", 1344 | "nbconvert_exporter": "python", 1345 | "pygments_lexer": "ipython3", 1346 | "version": "3.10.8" 1347 | } 1348 | }, 1349 | "nbformat": 4, 1350 | "nbformat_minor": 5 1351 | } 1352 | -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/mda_with_llm_langchain_byo_model.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0855b561", 6 | "metadata": {}, 7 | "source": [ 8 | "# Integrate Modern Data Architectures with Generative AI and interact using prompts for querying SQL databases & APIs" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "26105729-b3e3-42d0-a583-8446fff89277", 14 | "metadata": {}, 15 | "source": [ 16 | "This notebook demonstrates how **_large language models, such as Anthropic,_** interact with AWS databases, data stores, and third-party data warehousing solutions like Snowflake. We showcase this interaction 1) by generating and running SQL queries, and 2) making requests to API endpoints. We achieve all of this by using the LangChain framework, which allows the language model to interact with its environment and connect with other sources of data. The LangChain framework operates based on the following principles: calling out to a language model, being data-aware, and being agentic. Our notebook focuses on establishing database connections to various data sources, consolidating metadata, and returning fact-based data points in response to user queries using LLMs and LangChain." 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "a310d6ea-2ee1-4979-bb5e-b65cb892c0cd", 22 | "metadata": {}, 23 | "source": [ 24 | "\n", 25 | "\n" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "id": "5d0297e0-f2dd-464b-9254-6693c45ebafc", 31 | "metadata": { 32 | "tags": [] 33 | }, 34 | "source": [ 35 | "Step 1. Connection to various channels through which LLMs can talk to your data. These channels include:\n", 36 | "\n", 37 | " - RedShift Serverless - to connect to datastore 'tickit'(ticket is referred as tickit in the sample data store) to retrieve information regarding ticket sales.\n", 38 | " - Aurora - MySQL Serverless - to connect to datastore that hosts information about the employees.\n", 39 | " - S3/Athena - to connect to the SageMaker's offline feature store on claims information. \n", 40 | " - Snowflake - to connect to stocks related data residing in finance schema of 3rd party software.\n", 41 | " - APIs - to connect to meteo(in this example we use Langchain's sample dataset on meteo) to retrieve weather information.\n", 42 | " \n", 43 | "Step 2. Usage of Dynamic generation of prompt templates by populating metadata of the tables using Glue Data Catalog(GDC) as context. GDC was populated by running a crawler on the databases. Refer to the information here to create and run a glue crawler. In case of api, a line item was created in GDC data extract.\n", 44 | "\n", 45 | "Step 3. Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query\n", 46 | "\n", 47 | "Step 4. Apply user query to LLM and Langchain to determine the data channel. After determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 48 | "\n", 49 | "Finally, display the results.\n" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "id": "e0986ea2-f794-431f-a341-b94f0118cb7d", 55 | "metadata": { 56 | "tags": [] 57 | }, 58 | "source": [ 59 | "### Pre-requisites:\n", 60 | "1. Use kernel Base Python 3.0.\n", 61 | "2. Deploy resources using the cloudformation template mda-llm-cfn.yml.\n", 62 | "3. Access to the LLM API. In this notebook, Anthropic Model is used. Refer [here](https://console.anthropic.com/docs/access) for detais on how to get access to Anthropic API key.\n", 63 | "\n", 64 | "[OPTIONAL] - If need is to add any of the sources below, then uncomment code in the relevant sections.\n", 65 | "\n", 66 | "1. Setup [Aurora MySQL Serverless database](https://aws.amazon.com/getting-started/hands-on/building-serverless-applications-with-amazon-aurora-serverless/?ref=gsrchandson). Load sample dataset for Human Resource department. Use this notebook to load the data into Aurora MySQL.\n", 67 | "2. Setup [Redshift Serverless](https://catalog.workshops.aws/redshift-immersion/en-US/lab1). Load sample data for Sales & Marketing. For example, 'sample data dev' for 'tickit' dataset available in RedShift examples.\n", 68 | "3. Setup External database. In this case, we are using Snowflake account and populating stocks data. Use this notebook to load the data into Snowflake.\n", 69 | "4. Add/modify the [Glue Crawler](https://catalog.us-east-1.prod.workshops.aws/workshops/71b5bdcf-7eb1-4549-b851-66adc860cd04/en-US/2-studio/1-crawler) on all the databases mentioned above. \n", 70 | "\n", 71 | "**Note - This notebook was tested on kernel - conda_python3 in Region us-east-1**" 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": null, 77 | "id": "9556eddc-8e45-4e42-9157-213316ec468a", 78 | "metadata": { 79 | "tags": [] 80 | }, 81 | "outputs": [], 82 | "source": [ 83 | "%%writefile requirements.txt\n", 84 | "sqlalchemy==1.4.47\n", 85 | "snowflake-sqlalchemy\n", 86 | "langchain==0.0.166\n", 87 | "sqlalchemy-aurora-data-api\n", 88 | "PyAthena[SQLAlchemy]==2.25.2\n", 89 | "anthropic\n", 90 | "redshift-connector==2.0.910\n", 91 | "sqlalchemy-redshift==0.8.14" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "id": "b55d516c", 98 | "metadata": { 99 | "scrolled": true, 100 | "tags": [] 101 | }, 102 | "outputs": [], 103 | "source": [ 104 | "!pip install -r requirements.txt" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": null, 110 | "id": "91c153cd", 111 | "metadata": { 112 | "tags": [] 113 | }, 114 | "outputs": [], 115 | "source": [ 116 | "import json\n", 117 | "import boto3\n", 118 | "\n", 119 | "import sqlalchemy\n", 120 | "from sqlalchemy import create_engine\n", 121 | "from snowflake.sqlalchemy import URL\n", 122 | "\n", 123 | "from langchain.docstore.document import Document\n", 124 | "from langchain import PromptTemplate,SagemakerEndpoint,SQLDatabase, SQLDatabaseChain, LLMChain\n", 125 | "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n", 126 | "from langchain.chains.question_answering import load_qa_chain\n", 127 | "from langchain.prompts.prompt import PromptTemplate\n", 128 | "from langchain.chains import SQLDatabaseSequentialChain\n", 129 | "\n", 130 | "from langchain.chains.api.prompt import API_RESPONSE_PROMPT\n", 131 | "from langchain.chains import APIChain\n", 132 | "from langchain.prompts.prompt import PromptTemplate\n", 133 | "from langchain.chat_models import ChatAnthropic\n", 134 | "from langchain.chains.api import open_meteo_docs\n", 135 | "\n", 136 | "from typing import Dict" 137 | ] 138 | }, 139 | { 140 | "cell_type": "markdown", 141 | "id": "074a4144-4053-46c1-ba39-8f64f8fb9e00", 142 | "metadata": {}, 143 | "source": [ 144 | "The data for this COVID-19 dataset is stored in a public accessible S3 bucket. You can use the following command to explore the dataset.\n", 145 | "\n", 146 | "!aws s3 ls s3://covid19-lake/ --no-sign-request" 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "id": "87f05601-529a-42d4-8cab-ba0b9445e695", 152 | "metadata": { 153 | "tags": [] 154 | }, 155 | "source": [ 156 | "### Read parameters from Cloud Formation stack\n", 157 | "Some of the resources needed for this notebook such as the LLM model endpoint, the AWS Glue database and Glue crawler are created through a cloud formation template. The next block of code extracts the outputs and parameters of the cloud formation stack created from that template to get the value of these parameters.\n", 158 | "\n", 159 | "*The stack name here should match the stack name you used when creating the cloud formation stack.*" 160 | ] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "execution_count": null, 165 | "id": "406e3ede-99aa-46e0-b3aa-eb8b65f6cff5", 166 | "metadata": { 167 | "tags": [] 168 | }, 169 | "outputs": [], 170 | "source": [ 171 | "# if used a different name while creating the cloud formation stack then change this to match the name you used\n", 172 | "CFN_STACK_NAME = \"cfn-genai-mda\"" 173 | ] 174 | }, 175 | { 176 | "cell_type": "code", 177 | "execution_count": null, 178 | "id": "86a1efa5-a1ae-4384-b149-4103b70fab56", 179 | "metadata": { 180 | "tags": [] 181 | }, 182 | "outputs": [], 183 | "source": [ 184 | "stacks = boto3.client('cloudformation').list_stacks()\n", 185 | "stack_found = CFN_STACK_NAME in [stack['StackName'] for stack in stacks['StackSummaries']]" 186 | ] 187 | }, 188 | { 189 | "cell_type": "code", 190 | "execution_count": null, 191 | "id": "c0bda7a4-f2a9-4379-b4a9-8e3e17240d0f", 192 | "metadata": { 193 | "tags": [] 194 | }, 195 | "outputs": [], 196 | "source": [ 197 | "from typing import List\n", 198 | "def get_cfn_outputs(stackname: str) -> List:\n", 199 | " cfn = boto3.client('cloudformation')\n", 200 | " outputs = {}\n", 201 | " for output in cfn.describe_stacks(StackName=stackname)['Stacks'][0]['Outputs']:\n", 202 | " outputs[output['OutputKey']] = output['OutputValue']\n", 203 | " return outputs\n", 204 | "\n", 205 | "def get_cfn_parameters(stackname: str) -> List:\n", 206 | " cfn = boto3.client('cloudformation')\n", 207 | " params = {}\n", 208 | " for param in cfn.describe_stacks(StackName=stackname)['Stacks'][0]['Parameters']:\n", 209 | " params[param['ParameterKey']] = param['ParameterValue']\n", 210 | " return params\n", 211 | "\n", 212 | "if stack_found is True:\n", 213 | " outputs = get_cfn_outputs(CFN_STACK_NAME)\n", 214 | " params = get_cfn_parameters(CFN_STACK_NAME)\n", 215 | " glue_crawler_name = params['CFNCrawlerName']\n", 216 | " glue_database_name = params['CFNDatabaseName']\n", 217 | " glue_databucket_name = params['DataBucketName']\n", 218 | " region = outputs['Region']\n", 219 | " print(f\"cfn outputs={outputs}\\nparams={params}\")\n", 220 | "else:\n", 221 | " print(\"Recheck our cloudformation stack name\")" 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "id": "59c5c0b3-d32e-41ff-85fe-dc80586fa737", 227 | "metadata": { 228 | "tags": [] 229 | }, 230 | "source": [ 231 | "### Copy the sample dataset to your S3 bucket" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": null, 237 | "id": "ceedd61d-9c21-45b4-b35e-69e5cd047f11", 238 | "metadata": { 239 | "tags": [] 240 | }, 241 | "outputs": [], 242 | "source": [ 243 | "!aws s3 cp --recursive s3://covid19-lake/rearc-covid-19-testing-data/json/states_daily/ s3://{glue_databucket_name}/covid-dataset/" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "id": "7dbd2b73-cec1-4dbf-8a94-692eb04e7979", 249 | "metadata": { 250 | "tags": [] 251 | }, 252 | "source": [ 253 | "### Run the crawler" 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": null, 259 | "id": "5ccbd062-0918-4dcf-a4c2-45bfa81e56c7", 260 | "metadata": { 261 | "tags": [] 262 | }, 263 | "outputs": [], 264 | "source": [ 265 | "%%writefile python_glueworkshop.py\n", 266 | "import boto3\n", 267 | "import argparse\n", 268 | "import time\n", 269 | "\n", 270 | "argParser = argparse.ArgumentParser()\n", 271 | "argParser.add_argument(\"-c\", \"--glue_crawler_name\", help=\"script help\")\n", 272 | "args = argParser.parse_args()\n", 273 | "print(args.glue_crawler_name )\n", 274 | "client = boto3.client('glue')\n", 275 | "crawler_name=args.glue_crawler_name\n", 276 | "\n", 277 | "def get_crawler_status(crawler_name):\n", 278 | " # Create a Glue client\n", 279 | " glue_client = boto3.client('glue')\n", 280 | "\n", 281 | " # Get the crawler details\n", 282 | " response = glue_client.get_crawler(Name=crawler_name)\n", 283 | "\n", 284 | " # Extract the crawler state\n", 285 | " crawler_state = response['Crawler']['State']\n", 286 | "\n", 287 | " return crawler_state\n", 288 | "\n", 289 | "# This is the command to start the Crawler\n", 290 | "try:\n", 291 | " response = client.start_crawler(Name=crawler_name )\n", 292 | " print(\"Successfully started crawler. The crawler may take 2-5 mins to detect the schema.\")\n", 293 | "\n", 294 | " while True:\n", 295 | " # Get the crawler status\n", 296 | " status = get_crawler_status(crawler_name)\n", 297 | "\n", 298 | " # Print the crawler status\n", 299 | " print(f\"Crawler '{crawler_name}' status: {status}\")\n", 300 | "\n", 301 | " if status == 'READY': # Replace 'READY' with the desired completed state\n", 302 | " break # Exit the loop if the desired state is reached\n", 303 | "\n", 304 | " time.sleep(10) # Sleep for 10 seconds before checking the status again\n", 305 | " \n", 306 | "except:\n", 307 | " print(\"error in starting crawler. Check the logs for the error details.\")\n" 308 | ] 309 | }, 310 | { 311 | "cell_type": "markdown", 312 | "id": "d1392725-610f-43c1-8fd9-b5f52c4585b1", 313 | "metadata": {}, 314 | "source": [ 315 | "Execute the python script by passing the glue crawler name from the cloudformation stack output." 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": null, 321 | "id": "75d0fa47-1651-4cff-802a-22ae6438a09c", 322 | "metadata": { 323 | "tags": [] 324 | }, 325 | "outputs": [], 326 | "source": [ 327 | "!python python_glueworkshop.py -c {glue_crawler_name}" 328 | ] 329 | }, 330 | { 331 | "cell_type": "markdown", 332 | "id": "4132ffc3-6947-49b6-b627-fae3df870b88", 333 | "metadata": { 334 | "tags": [] 335 | }, 336 | "source": [ 337 | "Before proceeding to the next step, check the status of the crawler. It should change from RUNNING to READY. " 338 | ] 339 | }, 340 | { 341 | "cell_type": "markdown", 342 | "id": "b51d1d0e-33fb-46ca-b82f-6294ea867cae", 343 | "metadata": { 344 | "tags": [] 345 | }, 346 | "source": [ 347 | "### Step 1 - Connect to databases using SQL Alchemy. \n", 348 | "\n", 349 | "Under the hood, LangChain uses SQLAlchemy to connect to SQL databases. The SQLDatabaseChain can therefore be used with any SQL dialect supported by SQLAlchemy, \n", 350 | "such as MS SQL, MySQL, MariaDB, PostgreSQL, Oracle SQL, and SQLite. Please refer to the SQLAlchemy documentation for more information about requirements for connecting to your database. \n" 351 | ] 352 | }, 353 | { 354 | "cell_type": "markdown", 355 | "id": "e5f5ce28-9b33-4061-8655-2b297d5c24a2", 356 | "metadata": { 357 | "tags": [] 358 | }, 359 | "source": [ 360 | "**Important**: The code below establishes a database connection for data sources and Large Language Models. Please note that the solution will only work if the database connection for your sources is defined in the cell below. Please refer to the Pre-requisites section. If your use case requires data from Aurora MySQL alone, then please comment out other data sources. Furthermore, please update the cluster details and variables for Aurora MySQL accordingly." 361 | ] 362 | }, 363 | { 364 | "cell_type": "code", 365 | "execution_count": null, 366 | "id": "1583cade", 367 | "metadata": { 368 | "scrolled": true, 369 | "tags": [] 370 | }, 371 | "outputs": [], 372 | "source": [ 373 | "#define connections\n", 374 | "\n", 375 | "# collect credentials from Secrets Manager\n", 376 | "#Refer here on how to use AWS Secrets Manager - https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html\n", 377 | "client = boto3.client('secretsmanager')\n", 378 | "\n", 379 | "#LLM \n", 380 | "#get the llm api key\n", 381 | "#llm variables\n", 382 | "#Refer here for access to Anthropic API Keys https://console.anthropic.com/docs/access\n", 383 | "anthropic_secret_id = \"anthropic\"#\n", 384 | "## llm get credentials from secrets manager\n", 385 | "response = client.get_secret_value(SecretId=anthropic_secret_id)\n", 386 | "secrets_credentials = json.loads(response['SecretString'])\n", 387 | "ANTHROPIC_API_KEY = secrets_credentials['ANTHROPIC_API_KEY']\n", 388 | "#define large language model here. Make sure to set api keys for the variable ANTHROPIC_API_KEY\n", 389 | "llm = ChatAnthropic(temperature=0, anthropic_api_key=ANTHROPIC_API_KEY, max_tokens_to_sample = 512)\n", 390 | "\n", 391 | "#S3\n", 392 | "# connect to s3 using athena\n", 393 | "## athena variables\n", 394 | "connathena=f\"athena.{region}.amazonaws.com\" \n", 395 | "portathena='443' #Update, if port is different\n", 396 | "schemaathena=glue_database_name #from cfn params\n", 397 | "s3stagingathena=f's3://{glue_databucket_name}/athenaresults/'#from cfn params\n", 398 | "wkgrpathena='primary'#Update, if workgroup is different\n", 399 | "# tablesathena=['dataset']#[]\n", 400 | "## Create the athena connection string\n", 401 | "connection_string = f\"awsathena+rest://@{connathena}:{portathena}/{schemaathena}?s3_staging_dir={s3stagingathena}/&work_group={wkgrpathena}\"\n", 402 | "## Create the athena SQLAlchemy engine\n", 403 | "engine_athena = create_engine(connection_string, echo=False)\n", 404 | "dbathena = SQLDatabase(engine_athena)\n", 405 | "# dbathena = SQLDatabase(engine_athena, include_tables=tablesathena)\n", 406 | "# #SNOWFLAKE\n", 407 | "# # connect to snowflake database\n", 408 | "# ## snowflake variables\n", 409 | "# sf_account_id = \n", 410 | "# sf_secret_id =\n", 411 | "# dwh = \n", 412 | "# db = \n", 413 | "# schema = \n", 414 | "# table =
\n", 415 | "# ## snowflake get credentials from secrets manager\n", 416 | "# response = client.get_secret_value(SecretId=sf_secret_id)\n", 417 | "# secrets_credentials = json.loads(response['SecretString'])\n", 418 | "# sf_password = secrets_credentials['password']\n", 419 | "# sf_username = secrets_credentials['username']\n", 420 | "# ## Create the snowflake connection string\n", 421 | "# connection_string = f\"snowflake://{sf_username}:{sf_password}@{sf_account_id}/{db}/{schema}?warehouse={dwh}\"\n", 422 | "# ## Create the snowflake SQLAlchemy engine\n", 423 | "# engine_snowflake = create_engine(connection_string, echo=False)\n", 424 | "# dbsnowflake = SQLDatabase(engine_snowflake)\n", 425 | "\n", 426 | "# #AURORA MYSQL\n", 427 | "# ##connect to aurora mysql\n", 428 | "# ##aurora mysql cluster details/variables\n", 429 | "# cluster_arn = \n", 430 | "# secret_arn =\n", 431 | "# rdsdb=\n", 432 | "# rdsdb_tbl = [
]\n", 433 | "# ## Create the aurora connection string\n", 434 | "# connection_string = f\"mysql+auroradataapi://:@/{rdsdb}\"\n", 435 | "# ## Create the aurora SQLAlchemy engine\n", 436 | "# engine_rds = create_engine(connection_string, echo=False,connect_args=dict(aurora_cluster_arn=cluster_arn, secret_arn=secret_arn))\n", 437 | "# dbrds = SQLDatabase(engine_rds, include_tables=rdsdb_tbl)\n", 438 | "\n", 439 | "# #REDSHIFT\n", 440 | "# # connect to redshift database\n", 441 | "# ## redshift variables\n", 442 | "# rs_secret_id = \n", 443 | "# rs_endpoint=\n", 444 | "# rs_port=\n", 445 | "# rs_db=\n", 446 | "# rs_schema=\n", 447 | "# ## redshift get credentials from secrets manager\n", 448 | "# response = client.get_secret_value(SecretId=rs_secret_id)\n", 449 | "# secrets_credentials = json.loads(response['SecretString'])\n", 450 | "# rs_password = secrets_credentials['password']\n", 451 | "# rs_username = secrets_credentials['username']\n", 452 | "# ## Create the redshift connection string\n", 453 | "# connection_string = f\"redshift+redshift_connector://{rs_username}:{rs_password}@{rs_endpoint}:{rs_port}/{rs_db}\"\n", 454 | "# engine_redshift = create_engine(connection_string, echo=False)\n", 455 | "# dbredshift = SQLDatabase(engine_redshift)\n", 456 | "\n", 457 | "#Glue Data Catalog\n", 458 | "##Provide list of all the databases where the table metadata resides after the glue successfully crawls the table\n", 459 | "# gdc = ['redshift-sagemaker-sample-data-dev', 'snowflake','rds-aurora-mysql-employees','sagemaker_featurestore'] # mentioned a few examples here\n", 460 | "gdc = [schemaathena] \n" 461 | ] 462 | }, 463 | { 464 | "cell_type": "markdown", 465 | "id": "1ea21757-b08a-438b-a5a7-79d85a9a9085", 466 | "metadata": {}, 467 | "source": [ 468 | "### Step 2 - Generate Dynamic Prompt Templates\n", 469 | "Build a consolidated view of Glue Data Catalog by combining metadata stored for all the databases in pipe delimited format." 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": 11, 475 | "id": "08a3373d-9285-4fab-81b5-51e5364590b5", 476 | "metadata": { 477 | "scrolled": true, 478 | "tags": [] 479 | }, 480 | "outputs": [ 481 | { 482 | "name": "stdout", 483 | "output_type": "stream", 484 | "text": [ 485 | "s3|cfn_covid_lake|cfn_covid_dataset|totaltestresults\n", 486 | "s3|cfn_covid_lake|cfn_covid_dataset|fips\n", 487 | "s3|cfn_covid_lake|cfn_covid_dataset|deathincrease\n", 488 | "s3|cfn_covid_lake|cfn_covid_dataset|hospitalizedincrease\n", 489 | "s3|cfn_covid_lake|cfn_covid_dataset|negativeincrease\n", 490 | "s3|cfn_covid_lake|cfn_covid_dataset|positiveincrease\n", 491 | "s3|cfn_covid_lake|cfn_covid_dataset|totaltestresultsincrease\n", 492 | "s3|cfn_covid_lake|cfn_covid_dataset|negative\n", 493 | "s3|cfn_covid_lake|cfn_covid_dataset|pending\n", 494 | "api|meteo|weather|weather\n" 495 | ] 496 | } 497 | ], 498 | "source": [ 499 | "#Generate Dynamic prompts to populate the Glue Data Catalog\n", 500 | "#harvest aws crawler metadata\n", 501 | "\n", 502 | "def parse_catalog():\n", 503 | " #Connect to Glue catalog\n", 504 | " #get metadata of redshift serverless tables\n", 505 | " columns_str=''\n", 506 | " \n", 507 | " #define glue cient\n", 508 | " glue_client = boto3.client('glue')\n", 509 | " \n", 510 | " for db in gdc:\n", 511 | " response = glue_client.get_tables(DatabaseName =db)\n", 512 | " for tables in response['TableList']:\n", 513 | " #classification in the response for s3 and other databases is different. Set classification based on the response location\n", 514 | " if tables['StorageDescriptor']['Location'].startswith('s3'): classification='s3' \n", 515 | " else: classification = tables['Parameters']['classification']\n", 516 | " for columns in tables['StorageDescriptor']['Columns']:\n", 517 | " dbname,tblname,colname=tables['DatabaseName'],tables['Name'],columns['Name']\n", 518 | " columns_str=columns_str+f'\\n{classification}|{dbname}|{tblname}|{colname}' \n", 519 | " #API\n", 520 | " ## Append the metadata of the API to the unified glue data catalog\n", 521 | " columns_str=columns_str+'\\n'+('api|meteo|weather|weather')\n", 522 | " return columns_str\n", 523 | "\n", 524 | "glue_catalog = parse_catalog()\n", 525 | "\n", 526 | "#display a few lines from the catalog\n", 527 | "print('\\n'.join(glue_catalog.splitlines()[-10:]) )\n" 528 | ] 529 | }, 530 | { 531 | "cell_type": "markdown", 532 | "id": "a94e6770-42c3-402b-a60e-9c21fb99d5f6", 533 | "metadata": { 534 | "tags": [] 535 | }, 536 | "source": [ 537 | "### Step 3 - Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query" 538 | ] 539 | }, 540 | { 541 | "cell_type": "markdown", 542 | "id": "adda3714-3f32-4480-9526-91cca37489d1", 543 | "metadata": {}, 544 | "source": [ 545 | "In this code sample, we use the Anthropic Model to generate inferences. You can utilize SageMaker JumpStart models to achieve the same. \n", 546 | "Guidance on how to use the JumpStart Models is available in the notebook - mda_with_llm_langchain_smjumpstart_flant5xl" 547 | ] 548 | }, 549 | { 550 | "cell_type": "code", 551 | "execution_count": null, 552 | "id": "4efcc59b", 553 | "metadata": { 554 | "tags": [] 555 | }, 556 | "outputs": [], 557 | "source": [ 558 | "#Function 1 'Infer Channel'\n", 559 | "#define a function that infers the channel/database/table and sets the database for querying\n", 560 | "def identify_channel(query):\n", 561 | " #Prompt 1 'Infer Channel'\n", 562 | " ##set prompt template. It instructs the llm on how to evaluate and respond to the llm. It is referred to as dynamic since glue data catalog is first getting generated and appended to the prompt.\n", 563 | " prompt_template = \"\"\"\n", 564 | " From the table below, find the database (in column database) which will contain the data (in corresponding column_names) to answer the question \n", 565 | " {query} \\n\n", 566 | " \"\"\"+glue_catalog +\"\"\" \n", 567 | " Give your answer as database == \n", 568 | " Also,give your answer as database.table == \n", 569 | " \"\"\"\n", 570 | " ##define prompt 1\n", 571 | " PROMPT_channel = PromptTemplate( template=prompt_template, input_variables=[\"query\"] )\n", 572 | "\n", 573 | " # define llm chain\n", 574 | " llm_chain = LLMChain(prompt=PROMPT_channel, llm=llm)\n", 575 | " #run the query and save to generated texts\n", 576 | " generated_texts = llm_chain.run(query)\n", 577 | " print(generated_texts)\n", 578 | "\n", 579 | " #set the best channel from where the query can be answered\n", 580 | " if 'snowflake' in generated_texts: \n", 581 | " channel='db'\n", 582 | " db=dbsnowflake \n", 583 | " print(\"SET database to snowflake\") \n", 584 | " elif 'redshift' in generated_texts: \n", 585 | " channel='db'\n", 586 | " db=dbredshift\n", 587 | " print(\"SET database to redshift\")\n", 588 | " elif 's3' in generated_texts: \n", 589 | " channel='db'\n", 590 | " db=dbathena\n", 591 | " print(\"SET database to athena\")\n", 592 | " elif 'rdsmysql' in generated_texts: \n", 593 | " channel='db'\n", 594 | " db=dbrds\n", 595 | " print(\"SET database to rds\") \n", 596 | " elif 'api' in generated_texts: \n", 597 | " channel='api'\n", 598 | " print(\"SET database to weather api\") \n", 599 | " else: raise Exception(\"User question cannot be answered by any of the channels mentioned in the catalog\")\n", 600 | " print(\"Step complete. Channel is: \", channel)\n", 601 | " \n", 602 | " return channel, db\n", 603 | "\n", 604 | "#Function 2 'Run Query'\n", 605 | "#define a function that infers the channel/database/table and sets the database for querying\n", 606 | "def run_query(query):\n", 607 | "\n", 608 | " channel, db = identify_channel(query) #call the identify channel function first\n", 609 | "\n", 610 | " ##Prompt 2 'Run Query'\n", 611 | " #after determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 612 | " #provide rules for running the SQL queries in default template--> table info.\n", 613 | "\n", 614 | " _DEFAULT_TEMPLATE = \"\"\"Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.\n", 615 | "\n", 616 | " Do not append 'Query:' to SQLQuery.\n", 617 | " \n", 618 | " Display SQLResult after the query is run in plain english that users can understand. \n", 619 | "\n", 620 | " Provide answer in simple english statement.\n", 621 | " \n", 622 | " Only use the following tables:\n", 623 | "\n", 624 | " {table_info}\n", 625 | " If someone asks for the sales, they really mean the tickit.sales table.\n", 626 | " If someone asks for the sales date, they really mean the column tickit.sales.saletime.\n", 627 | "\n", 628 | " Question: {input}\"\"\"\n", 629 | "\n", 630 | " PROMPT_sql = PromptTemplate(\n", 631 | " input_variables=[\"input\", \"table_info\", \"dialect\"], template=_DEFAULT_TEMPLATE\n", 632 | " )\n", 633 | "\n", 634 | " \n", 635 | " if channel=='db':\n", 636 | " db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT_sql, verbose=True, return_intermediate_steps=False)\n", 637 | " response=db_chain.run(query)\n", 638 | " elif channel=='api':\n", 639 | " chain_api = APIChain.from_llm_and_api_docs(llm, open_meteo_docs.OPEN_METEO_DOCS, verbose=True)\n", 640 | " response=chain_api.run(query)\n", 641 | " else: raise Exception(\"Unlisted channel. Check your unified catalog\")\n", 642 | " return response\n", 643 | "\n" 644 | ] 645 | }, 646 | { 647 | "cell_type": "markdown", 648 | "id": "390a92cd-e1b4-4feb-ab7a-f97030ba7f84", 649 | "metadata": {}, 650 | "source": [ 651 | "### Step 4 - Run the run_query function that in turn calls the Langchain SQL Database chain to convert 'text to sql' and runs the query against the source data channel\n", 652 | "\n", 653 | "Some samples are provided below for test runs. Uncomment the query to run." 654 | ] 655 | }, 656 | { 657 | "cell_type": "code", 658 | "execution_count": 13, 659 | "id": "f82599a2", 660 | "metadata": { 661 | "tags": [] 662 | }, 663 | "outputs": [ 664 | { 665 | "name": "stdout", 666 | "output_type": "stream", 667 | "text": [ 668 | " database == s3\n", 669 | "database.table == cfn_covid_lake.cfn_covid_dataset\n", 670 | "SET database to athena\n", 671 | "Step complete. Channel is: db\n", 672 | "\n", 673 | "\n", 674 | "\u001b[1m> Entering new SQLDatabaseChain chain...\u001b[0m\n", 675 | "Which States reported the least and maximum deaths?\n", 676 | "SQLQuery:\u001b[32;1m\u001b[1;3m SELECT state, MIN(death) AS least_deaths, MAX(death) AS max_deaths \n", 677 | "FROM cfn_covid_dataset \n", 678 | "GROUP BY state \n", 679 | "ORDER BY least_deaths, max_deaths\n", 680 | "\u001b[0m\n", 681 | "SQLResult: \u001b[33;1m\u001b[1;3m[('AS', 0.0, 0.0), ('MP', 0.0, 2.0), ('AK', 0.0, 305.0), ('WY', 0.0, 682.0), ('DC', 0.0, 1030.0), ('DE', 0.0, 1473.0), ('ND', 0.0, 1478.0), ('ID', 0.0, 1879.0), ('UT', 0.0, 1976.0), ('NE', 0.0, 2113.0), ('WV', 0.0, 2325.0), ('NV', 0.0, 5037.0), ('AR', 0.0, 5417.0), ('MO', 0.0, 8161.0), ('AL', 0.0, 10149.0), ('NC', 0.0, 11502.0), ('IN', 0.0, 12737.0), ('AZ', 0.0, 16328.0), ('MI', 0.0, 16658.0), ('NJ', 0.0, 23574.0), ('VI', 1.0, 25.0), ('GU', 1.0, 133.0), ('HI', 1.0, 445.0), ('ME', 1.0, 706.0), ('NH', 1.0, 1184.0), ('MT', 1.0, 1381.0), ('SD', 1.0, 1900.0), ('PR', 1.0, 2059.0), ('NM', 1.0, 3808.0), ('OK', 1.0, 4534.0), ('KS', 1.0, 4816.0), ('KY', 1.0, 4819.0), ('IA', 1.0, 5558.0), ('CO', 1.0, 5989.0), ('MN', 1.0, 6550.0), ('MS', 1.0, 6808.0), ('CT', 1.0, 7704.0), ('SC', 1.0, 8754.0), ('VA', 1.0, 9596.0), ('OH', 1.0, 17656.0), ('GA', 1.0, 17906.0), ('IL', 1.0, 23014.0), ('PA', 1.0, 24349.0), ('TX', 1.0, 44451.0), ('VT', 2.0, 208.0), ('WA', 2.0, 5041.0), ('MD', 2.0, 7955.0), ('LA', 2.0, 9748.0), ('TN', 2.0, 11543.0), ('MA', 2.0, 16417.0), ('FL', 2.0, 32266.0), ('OR', 3.0, 2296.0), ('RI', 3.0, 2541.0), ('WI', 3.0, 7106.0), ('NY', 3.0, 39029.0), ('CA', 4.0, 54124.0)]\u001b[0m\n", 682 | "Answer:\u001b[32;1m\u001b[1;3m The states that reported the least deaths are American Samoa, Northern Mariana Islands and Alaska. \n", 683 | "The states that reported the maximum deaths are California, New York and Florida.\u001b[0m\n", 684 | "\u001b[1m> Finished chain.\u001b[0m\n", 685 | "----------------------------------------------------------------------\n", 686 | "SQL and response from user query Which States reported the least and maximum deaths? \n", 687 | " The states that reported the least deaths are American Samoa, Northern Mariana Islands and Alaska. \n", 688 | "The states that reported the maximum deaths are California, New York and Florida.\n" 689 | ] 690 | } 691 | ], 692 | "source": [ 693 | "# Enter the query\n", 694 | "## Few queries to try out - \n", 695 | "#athena - Healthcare - Covid dataset\n", 696 | "# query = \"\"\"How many covid hospitalizations were reported in NY in June of 2021?\"\"\" \n", 697 | "query = \"\"\"Which States reported the least and maximum deaths?\"\"\" \n", 698 | "\n", 699 | "#snowflake - Finance and Investments\n", 700 | "# query = \"\"\"Which stock performed the best and the worst in May of 2013?\"\"\"\n", 701 | "# query = \"\"\"What is the average volume stocks traded in July of 2013?\"\"\"\n", 702 | "\n", 703 | "#rds - Human Resources\n", 704 | "# query = \"\"\"Name all employees with birth date this month\"\"\" \n", 705 | "# query = \"\"\"Combien d'employés sont des femmes? \"\"\" #Ask question in French - How many females are there?\n", 706 | "# query = \"\"\"How many employees were hired before 1990?\"\"\" \n", 707 | "\n", 708 | "#athena - Legal - SageMaker offline featurestore\n", 709 | "# query = \"\"\"How many frauds happened in the year 2023 ?\"\"\" \n", 710 | "# query = \"\"\"How many policies were claimed this year ?\"\"\" \n", 711 | "\n", 712 | "#redshift - Sales & Marketing\n", 713 | "# query = \"\"\"How many tickit sales are there\"\"\" \n", 714 | "# query = \"what was the total commision for the tickit sales in the year 2008?\" \n", 715 | "\n", 716 | "#api - product - weather\n", 717 | "# query = \"\"\"What is the weather like right now in New York City in degrees Farenheit?\"\"\"\n", 718 | "\n", 719 | "#Response from Langchain\n", 720 | "response = run_query(query)\n", 721 | "print(\"----------------------------------------------------------------------\")\n", 722 | "print(f'SQL and response from user query {query} \\n {response}')" 723 | ] 724 | }, 725 | { 726 | "cell_type": "markdown", 727 | "id": "69371bdc-537f-4e5e-a004-99d852097862", 728 | "metadata": {}, 729 | "source": [ 730 | "### Clean-up\n", 731 | "After you run the modern data architecture with Generative AI, make sure to clean up any resources that won’t be utilized. Shutdown and delete the databases used (Amazon Redshift, Amazon RDS, Snowflake). In addition, delete the data in Amazon S3 and make sure to stop any SageMaker Studio notebook instances to not incur any further charges. If you used SageMaker Jumpstart to deploy large language model as SageMaker Real-time Endpoint, delete endpoint either through SageMaker console, or through Studio. \n", 732 | "\n", 733 | "To completely remove all the provisoned resources, go to CloudFormation and delete the stack.\n" 734 | ] 735 | } 736 | ], 737 | "metadata": { 738 | "availableInstances": [ 739 | { 740 | "_defaultOrder": 0, 741 | "_isFastLaunch": true, 742 | "category": "General purpose", 743 | "gpuNum": 0, 744 | "hideHardwareSpecs": false, 745 | "memoryGiB": 4, 746 | "name": "ml.t3.medium", 747 | "vcpuNum": 2 748 | }, 749 | { 750 | "_defaultOrder": 1, 751 | "_isFastLaunch": false, 752 | "category": "General purpose", 753 | "gpuNum": 0, 754 | "hideHardwareSpecs": false, 755 | "memoryGiB": 8, 756 | "name": "ml.t3.large", 757 | "vcpuNum": 2 758 | }, 759 | { 760 | "_defaultOrder": 2, 761 | "_isFastLaunch": false, 762 | "category": "General purpose", 763 | "gpuNum": 0, 764 | "hideHardwareSpecs": false, 765 | "memoryGiB": 16, 766 | "name": "ml.t3.xlarge", 767 | "vcpuNum": 4 768 | }, 769 | { 770 | "_defaultOrder": 3, 771 | "_isFastLaunch": false, 772 | "category": "General purpose", 773 | "gpuNum": 0, 774 | "hideHardwareSpecs": false, 775 | "memoryGiB": 32, 776 | "name": "ml.t3.2xlarge", 777 | "vcpuNum": 8 778 | }, 779 | { 780 | "_defaultOrder": 4, 781 | "_isFastLaunch": true, 782 | "category": "General purpose", 783 | "gpuNum": 0, 784 | "hideHardwareSpecs": false, 785 | "memoryGiB": 8, 786 | "name": "ml.m5.large", 787 | "vcpuNum": 2 788 | }, 789 | { 790 | "_defaultOrder": 5, 791 | "_isFastLaunch": false, 792 | "category": "General purpose", 793 | "gpuNum": 0, 794 | "hideHardwareSpecs": false, 795 | "memoryGiB": 16, 796 | "name": "ml.m5.xlarge", 797 | "vcpuNum": 4 798 | }, 799 | { 800 | "_defaultOrder": 6, 801 | "_isFastLaunch": false, 802 | "category": "General purpose", 803 | "gpuNum": 0, 804 | "hideHardwareSpecs": false, 805 | "memoryGiB": 32, 806 | "name": "ml.m5.2xlarge", 807 | "vcpuNum": 8 808 | }, 809 | { 810 | "_defaultOrder": 7, 811 | "_isFastLaunch": false, 812 | "category": "General purpose", 813 | "gpuNum": 0, 814 | "hideHardwareSpecs": false, 815 | "memoryGiB": 64, 816 | "name": "ml.m5.4xlarge", 817 | "vcpuNum": 16 818 | }, 819 | { 820 | "_defaultOrder": 8, 821 | "_isFastLaunch": false, 822 | "category": "General purpose", 823 | "gpuNum": 0, 824 | "hideHardwareSpecs": false, 825 | "memoryGiB": 128, 826 | "name": "ml.m5.8xlarge", 827 | "vcpuNum": 32 828 | }, 829 | { 830 | "_defaultOrder": 9, 831 | "_isFastLaunch": false, 832 | "category": "General purpose", 833 | "gpuNum": 0, 834 | "hideHardwareSpecs": false, 835 | "memoryGiB": 192, 836 | "name": "ml.m5.12xlarge", 837 | "vcpuNum": 48 838 | }, 839 | { 840 | "_defaultOrder": 10, 841 | "_isFastLaunch": false, 842 | "category": "General purpose", 843 | "gpuNum": 0, 844 | "hideHardwareSpecs": false, 845 | "memoryGiB": 256, 846 | "name": "ml.m5.16xlarge", 847 | "vcpuNum": 64 848 | }, 849 | { 850 | "_defaultOrder": 11, 851 | "_isFastLaunch": false, 852 | "category": "General purpose", 853 | "gpuNum": 0, 854 | "hideHardwareSpecs": false, 855 | "memoryGiB": 384, 856 | "name": "ml.m5.24xlarge", 857 | "vcpuNum": 96 858 | }, 859 | { 860 | "_defaultOrder": 12, 861 | "_isFastLaunch": false, 862 | "category": "General purpose", 863 | "gpuNum": 0, 864 | "hideHardwareSpecs": false, 865 | "memoryGiB": 8, 866 | "name": "ml.m5d.large", 867 | "vcpuNum": 2 868 | }, 869 | { 870 | "_defaultOrder": 13, 871 | "_isFastLaunch": false, 872 | "category": "General purpose", 873 | "gpuNum": 0, 874 | "hideHardwareSpecs": false, 875 | "memoryGiB": 16, 876 | "name": "ml.m5d.xlarge", 877 | "vcpuNum": 4 878 | }, 879 | { 880 | "_defaultOrder": 14, 881 | "_isFastLaunch": false, 882 | "category": "General purpose", 883 | "gpuNum": 0, 884 | "hideHardwareSpecs": false, 885 | "memoryGiB": 32, 886 | "name": "ml.m5d.2xlarge", 887 | "vcpuNum": 8 888 | }, 889 | { 890 | "_defaultOrder": 15, 891 | "_isFastLaunch": false, 892 | "category": "General purpose", 893 | "gpuNum": 0, 894 | "hideHardwareSpecs": false, 895 | "memoryGiB": 64, 896 | "name": "ml.m5d.4xlarge", 897 | "vcpuNum": 16 898 | }, 899 | { 900 | "_defaultOrder": 16, 901 | "_isFastLaunch": false, 902 | "category": "General purpose", 903 | "gpuNum": 0, 904 | "hideHardwareSpecs": false, 905 | "memoryGiB": 128, 906 | "name": "ml.m5d.8xlarge", 907 | "vcpuNum": 32 908 | }, 909 | { 910 | "_defaultOrder": 17, 911 | "_isFastLaunch": false, 912 | "category": "General purpose", 913 | "gpuNum": 0, 914 | "hideHardwareSpecs": false, 915 | "memoryGiB": 192, 916 | "name": "ml.m5d.12xlarge", 917 | "vcpuNum": 48 918 | }, 919 | { 920 | "_defaultOrder": 18, 921 | "_isFastLaunch": false, 922 | "category": "General purpose", 923 | "gpuNum": 0, 924 | "hideHardwareSpecs": false, 925 | "memoryGiB": 256, 926 | "name": "ml.m5d.16xlarge", 927 | "vcpuNum": 64 928 | }, 929 | { 930 | "_defaultOrder": 19, 931 | "_isFastLaunch": false, 932 | "category": "General purpose", 933 | "gpuNum": 0, 934 | "hideHardwareSpecs": false, 935 | "memoryGiB": 384, 936 | "name": "ml.m5d.24xlarge", 937 | "vcpuNum": 96 938 | }, 939 | { 940 | "_defaultOrder": 20, 941 | "_isFastLaunch": false, 942 | "category": "General purpose", 943 | "gpuNum": 0, 944 | "hideHardwareSpecs": true, 945 | "memoryGiB": 0, 946 | "name": "ml.geospatial.interactive", 947 | "supportedImageNames": [ 948 | "sagemaker-geospatial-v1-0" 949 | ], 950 | "vcpuNum": 0 951 | }, 952 | { 953 | "_defaultOrder": 21, 954 | "_isFastLaunch": true, 955 | "category": "Compute optimized", 956 | "gpuNum": 0, 957 | "hideHardwareSpecs": false, 958 | "memoryGiB": 4, 959 | "name": "ml.c5.large", 960 | "vcpuNum": 2 961 | }, 962 | { 963 | "_defaultOrder": 22, 964 | "_isFastLaunch": false, 965 | "category": "Compute optimized", 966 | "gpuNum": 0, 967 | "hideHardwareSpecs": false, 968 | "memoryGiB": 8, 969 | "name": "ml.c5.xlarge", 970 | "vcpuNum": 4 971 | }, 972 | { 973 | "_defaultOrder": 23, 974 | "_isFastLaunch": false, 975 | "category": "Compute optimized", 976 | "gpuNum": 0, 977 | "hideHardwareSpecs": false, 978 | "memoryGiB": 16, 979 | "name": "ml.c5.2xlarge", 980 | "vcpuNum": 8 981 | }, 982 | { 983 | "_defaultOrder": 24, 984 | "_isFastLaunch": false, 985 | "category": "Compute optimized", 986 | "gpuNum": 0, 987 | "hideHardwareSpecs": false, 988 | "memoryGiB": 32, 989 | "name": "ml.c5.4xlarge", 990 | "vcpuNum": 16 991 | }, 992 | { 993 | "_defaultOrder": 25, 994 | "_isFastLaunch": false, 995 | "category": "Compute optimized", 996 | "gpuNum": 0, 997 | "hideHardwareSpecs": false, 998 | "memoryGiB": 72, 999 | "name": "ml.c5.9xlarge", 1000 | "vcpuNum": 36 1001 | }, 1002 | { 1003 | "_defaultOrder": 26, 1004 | "_isFastLaunch": false, 1005 | "category": "Compute optimized", 1006 | "gpuNum": 0, 1007 | "hideHardwareSpecs": false, 1008 | "memoryGiB": 96, 1009 | "name": "ml.c5.12xlarge", 1010 | "vcpuNum": 48 1011 | }, 1012 | { 1013 | "_defaultOrder": 27, 1014 | "_isFastLaunch": false, 1015 | "category": "Compute optimized", 1016 | "gpuNum": 0, 1017 | "hideHardwareSpecs": false, 1018 | "memoryGiB": 144, 1019 | "name": "ml.c5.18xlarge", 1020 | "vcpuNum": 72 1021 | }, 1022 | { 1023 | "_defaultOrder": 28, 1024 | "_isFastLaunch": false, 1025 | "category": "Compute optimized", 1026 | "gpuNum": 0, 1027 | "hideHardwareSpecs": false, 1028 | "memoryGiB": 192, 1029 | "name": "ml.c5.24xlarge", 1030 | "vcpuNum": 96 1031 | }, 1032 | { 1033 | "_defaultOrder": 29, 1034 | "_isFastLaunch": true, 1035 | "category": "Accelerated computing", 1036 | "gpuNum": 1, 1037 | "hideHardwareSpecs": false, 1038 | "memoryGiB": 16, 1039 | "name": "ml.g4dn.xlarge", 1040 | "vcpuNum": 4 1041 | }, 1042 | { 1043 | "_defaultOrder": 30, 1044 | "_isFastLaunch": false, 1045 | "category": "Accelerated computing", 1046 | "gpuNum": 1, 1047 | "hideHardwareSpecs": false, 1048 | "memoryGiB": 32, 1049 | "name": "ml.g4dn.2xlarge", 1050 | "vcpuNum": 8 1051 | }, 1052 | { 1053 | "_defaultOrder": 31, 1054 | "_isFastLaunch": false, 1055 | "category": "Accelerated computing", 1056 | "gpuNum": 1, 1057 | "hideHardwareSpecs": false, 1058 | "memoryGiB": 64, 1059 | "name": "ml.g4dn.4xlarge", 1060 | "vcpuNum": 16 1061 | }, 1062 | { 1063 | "_defaultOrder": 32, 1064 | "_isFastLaunch": false, 1065 | "category": "Accelerated computing", 1066 | "gpuNum": 1, 1067 | "hideHardwareSpecs": false, 1068 | "memoryGiB": 128, 1069 | "name": "ml.g4dn.8xlarge", 1070 | "vcpuNum": 32 1071 | }, 1072 | { 1073 | "_defaultOrder": 33, 1074 | "_isFastLaunch": false, 1075 | "category": "Accelerated computing", 1076 | "gpuNum": 4, 1077 | "hideHardwareSpecs": false, 1078 | "memoryGiB": 192, 1079 | "name": "ml.g4dn.12xlarge", 1080 | "vcpuNum": 48 1081 | }, 1082 | { 1083 | "_defaultOrder": 34, 1084 | "_isFastLaunch": false, 1085 | "category": "Accelerated computing", 1086 | "gpuNum": 1, 1087 | "hideHardwareSpecs": false, 1088 | "memoryGiB": 256, 1089 | "name": "ml.g4dn.16xlarge", 1090 | "vcpuNum": 64 1091 | }, 1092 | { 1093 | "_defaultOrder": 35, 1094 | "_isFastLaunch": false, 1095 | "category": "Accelerated computing", 1096 | "gpuNum": 1, 1097 | "hideHardwareSpecs": false, 1098 | "memoryGiB": 61, 1099 | "name": "ml.p3.2xlarge", 1100 | "vcpuNum": 8 1101 | }, 1102 | { 1103 | "_defaultOrder": 36, 1104 | "_isFastLaunch": false, 1105 | "category": "Accelerated computing", 1106 | "gpuNum": 4, 1107 | "hideHardwareSpecs": false, 1108 | "memoryGiB": 244, 1109 | "name": "ml.p3.8xlarge", 1110 | "vcpuNum": 32 1111 | }, 1112 | { 1113 | "_defaultOrder": 37, 1114 | "_isFastLaunch": false, 1115 | "category": "Accelerated computing", 1116 | "gpuNum": 8, 1117 | "hideHardwareSpecs": false, 1118 | "memoryGiB": 488, 1119 | "name": "ml.p3.16xlarge", 1120 | "vcpuNum": 64 1121 | }, 1122 | { 1123 | "_defaultOrder": 38, 1124 | "_isFastLaunch": false, 1125 | "category": "Accelerated computing", 1126 | "gpuNum": 8, 1127 | "hideHardwareSpecs": false, 1128 | "memoryGiB": 768, 1129 | "name": "ml.p3dn.24xlarge", 1130 | "vcpuNum": 96 1131 | }, 1132 | { 1133 | "_defaultOrder": 39, 1134 | "_isFastLaunch": false, 1135 | "category": "Memory Optimized", 1136 | "gpuNum": 0, 1137 | "hideHardwareSpecs": false, 1138 | "memoryGiB": 16, 1139 | "name": "ml.r5.large", 1140 | "vcpuNum": 2 1141 | }, 1142 | { 1143 | "_defaultOrder": 40, 1144 | "_isFastLaunch": false, 1145 | "category": "Memory Optimized", 1146 | "gpuNum": 0, 1147 | "hideHardwareSpecs": false, 1148 | "memoryGiB": 32, 1149 | "name": "ml.r5.xlarge", 1150 | "vcpuNum": 4 1151 | }, 1152 | { 1153 | "_defaultOrder": 41, 1154 | "_isFastLaunch": false, 1155 | "category": "Memory Optimized", 1156 | "gpuNum": 0, 1157 | "hideHardwareSpecs": false, 1158 | "memoryGiB": 64, 1159 | "name": "ml.r5.2xlarge", 1160 | "vcpuNum": 8 1161 | }, 1162 | { 1163 | "_defaultOrder": 42, 1164 | "_isFastLaunch": false, 1165 | "category": "Memory Optimized", 1166 | "gpuNum": 0, 1167 | "hideHardwareSpecs": false, 1168 | "memoryGiB": 128, 1169 | "name": "ml.r5.4xlarge", 1170 | "vcpuNum": 16 1171 | }, 1172 | { 1173 | "_defaultOrder": 43, 1174 | "_isFastLaunch": false, 1175 | "category": "Memory Optimized", 1176 | "gpuNum": 0, 1177 | "hideHardwareSpecs": false, 1178 | "memoryGiB": 256, 1179 | "name": "ml.r5.8xlarge", 1180 | "vcpuNum": 32 1181 | }, 1182 | { 1183 | "_defaultOrder": 44, 1184 | "_isFastLaunch": false, 1185 | "category": "Memory Optimized", 1186 | "gpuNum": 0, 1187 | "hideHardwareSpecs": false, 1188 | "memoryGiB": 384, 1189 | "name": "ml.r5.12xlarge", 1190 | "vcpuNum": 48 1191 | }, 1192 | { 1193 | "_defaultOrder": 45, 1194 | "_isFastLaunch": false, 1195 | "category": "Memory Optimized", 1196 | "gpuNum": 0, 1197 | "hideHardwareSpecs": false, 1198 | "memoryGiB": 512, 1199 | "name": "ml.r5.16xlarge", 1200 | "vcpuNum": 64 1201 | }, 1202 | { 1203 | "_defaultOrder": 46, 1204 | "_isFastLaunch": false, 1205 | "category": "Memory Optimized", 1206 | "gpuNum": 0, 1207 | "hideHardwareSpecs": false, 1208 | "memoryGiB": 768, 1209 | "name": "ml.r5.24xlarge", 1210 | "vcpuNum": 96 1211 | }, 1212 | { 1213 | "_defaultOrder": 47, 1214 | "_isFastLaunch": false, 1215 | "category": "Accelerated computing", 1216 | "gpuNum": 1, 1217 | "hideHardwareSpecs": false, 1218 | "memoryGiB": 16, 1219 | "name": "ml.g5.xlarge", 1220 | "vcpuNum": 4 1221 | }, 1222 | { 1223 | "_defaultOrder": 48, 1224 | "_isFastLaunch": false, 1225 | "category": "Accelerated computing", 1226 | "gpuNum": 1, 1227 | "hideHardwareSpecs": false, 1228 | "memoryGiB": 32, 1229 | "name": "ml.g5.2xlarge", 1230 | "vcpuNum": 8 1231 | }, 1232 | { 1233 | "_defaultOrder": 49, 1234 | "_isFastLaunch": false, 1235 | "category": "Accelerated computing", 1236 | "gpuNum": 1, 1237 | "hideHardwareSpecs": false, 1238 | "memoryGiB": 64, 1239 | "name": "ml.g5.4xlarge", 1240 | "vcpuNum": 16 1241 | }, 1242 | { 1243 | "_defaultOrder": 50, 1244 | "_isFastLaunch": false, 1245 | "category": "Accelerated computing", 1246 | "gpuNum": 1, 1247 | "hideHardwareSpecs": false, 1248 | "memoryGiB": 128, 1249 | "name": "ml.g5.8xlarge", 1250 | "vcpuNum": 32 1251 | }, 1252 | { 1253 | "_defaultOrder": 51, 1254 | "_isFastLaunch": false, 1255 | "category": "Accelerated computing", 1256 | "gpuNum": 1, 1257 | "hideHardwareSpecs": false, 1258 | "memoryGiB": 256, 1259 | "name": "ml.g5.16xlarge", 1260 | "vcpuNum": 64 1261 | }, 1262 | { 1263 | "_defaultOrder": 52, 1264 | "_isFastLaunch": false, 1265 | "category": "Accelerated computing", 1266 | "gpuNum": 4, 1267 | "hideHardwareSpecs": false, 1268 | "memoryGiB": 192, 1269 | "name": "ml.g5.12xlarge", 1270 | "vcpuNum": 48 1271 | }, 1272 | { 1273 | "_defaultOrder": 53, 1274 | "_isFastLaunch": false, 1275 | "category": "Accelerated computing", 1276 | "gpuNum": 4, 1277 | "hideHardwareSpecs": false, 1278 | "memoryGiB": 384, 1279 | "name": "ml.g5.24xlarge", 1280 | "vcpuNum": 96 1281 | }, 1282 | { 1283 | "_defaultOrder": 54, 1284 | "_isFastLaunch": false, 1285 | "category": "Accelerated computing", 1286 | "gpuNum": 8, 1287 | "hideHardwareSpecs": false, 1288 | "memoryGiB": 768, 1289 | "name": "ml.g5.48xlarge", 1290 | "vcpuNum": 192 1291 | } 1292 | ], 1293 | "instance_type": "ml.t3.medium", 1294 | "kernelspec": { 1295 | "display_name": "conda_python3", 1296 | "language": "python", 1297 | "name": "conda_python3" 1298 | }, 1299 | "language_info": { 1300 | "codemirror_mode": { 1301 | "name": "ipython", 1302 | "version": 3 1303 | }, 1304 | "file_extension": ".py", 1305 | "mimetype": "text/x-python", 1306 | "name": "python", 1307 | "nbconvert_exporter": "python", 1308 | "pygments_lexer": "ipython3", 1309 | "version": "3.10.8" 1310 | } 1311 | }, 1312 | "nbformat": 4, 1313 | "nbformat_minor": 5 1314 | } 1315 | -------------------------------------------------------------------------------- /blogs/Simple-text-to-sql/mda_text-to-sql_with_llm_langchain_bedrock.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0855b561", 6 | "metadata": {}, 7 | "source": [ 8 | "# Integrate Modern Data Architectures with Generative AI and interact using prompts for querying SQL databases & APIs" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "26105729-b3e3-42d0-a583-8446fff89277", 14 | "metadata": {}, 15 | "source": [ 16 | "This notebook demonstrates how **large language models, such as Amazon Titan and Claude Anthropic, accessible via [Amazon BedRock](https://aws.amazon.com/bedrock/)** interact with AWS databases, data stores, and third-party data warehousing solutions like Snowflake. We showcase this interaction 1) by generating and running SQL queries, and 2) making requests to API endpoints. We achieve all of this by using the LangChain framework, which allows the language model to interact with its environment and connect with other sources of data. The LangChain framework operates based on the following principles: calling out to a language model, being data-aware, and being agentic. " 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "d02c8cc5-5104-44aa-bbce-ad3ca7562a29", 22 | "metadata": { 23 | "tags": [] 24 | }, 25 | "source": [ 26 | "This notebook focuses on establishing connection to one data source, consolidating metadata, and returning fact-based data points in response to user queries using LLMs and LangChain. The solution can be enhanced to add multiple data sources." 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "id": "a310d6ea-2ee1-4979-bb5e-b65cb892c0cd", 32 | "metadata": {}, 33 | "source": [ 34 | "\n", 35 | "\n" 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "id": "e0986ea2-f794-431f-a341-b94f0118cb7d", 41 | "metadata": { 42 | "tags": [] 43 | }, 44 | "source": [ 45 | "### Pre-requisites:\n", 46 | "1. Use kernel Base Python 3.0.\n", 47 | "2. Install the required packages.\n", 48 | "3. Run the One time Setup by entering the user input parameters, copying the dataset, setup IAM role and finally run the crawler.\n", 49 | "3. Access to the LLM API. In this notebook, Anthropic Model is used. Refer [here](https://console.anthropic.com/docs/access) for detais on how to get access to Anthropic API key.\n", 50 | "\n", 51 | "**Note - This notebook was tested on kernel - conda_python3 in Region us-east-1**" 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "id": "9597c6f9", 57 | "metadata": {}, 58 | "source": [ 59 | "1. Attach AmazonAthenaFullAccess, AWSGlueServiceRole in IAM.\n", 60 | "2. Add the following custom policy in IAM to grant creating policy (double click cell to get json format)." 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "id": "4f9731fb", 66 | "metadata": {}, 67 | "source": [ 68 | "{\n", 69 | " \"Version\": \"2012-10-17\",\n", 70 | " \"Statement\": [\n", 71 | " {\n", 72 | " \"Action\": [\n", 73 | " \"iam:AttachRolePolicy\",\n", 74 | " \"iam:CreateRole\",\n", 75 | " \"iam:CreatePolicy\",\n", 76 | " \"iam:GetRole\",\n", 77 | " \"iam:PassRole\"\n", 78 | " ],\n", 79 | " \"Effect\": \"Allow\",\n", 80 | " \"Resource\": \"*\"\n", 81 | " }\n", 82 | " ]\n", 83 | "}" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "id": "5d0297e0-f2dd-464b-9254-6693c45ebafc", 89 | "metadata": { 90 | "tags": [] 91 | }, 92 | "source": [ 93 | "### Solution Walkthrough:\n", 94 | "\n", 95 | "Step 1. Connection to S3 through which LLMs can talk to your data. These channels include:\n", 96 | " - S3/Athena - to connect to the SageMaker's offline feature store on claims information. \n", 97 | " \n", 98 | "Step 2. Usage of Dynamic generation of prompt templates by populating metadata of the tables using Glue Data Catalog(GDC) as context. GDC was populated by running a crawler on the databases. Refer to the information here to create and run a glue crawler. In case of api, a line item was created in GDC data extract.\n", 99 | "\n", 100 | "Step 3. Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query\n", 101 | "\n", 102 | "Step 4. Apply user query to LLM and Langchain to determine the data channel. After determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 103 | "\n", 104 | "Finally, display the results.\n" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": 1, 110 | "id": "9556eddc-8e45-4e42-9157-213316ec468a", 111 | "metadata": { 112 | "tags": [] 113 | }, 114 | "outputs": [ 115 | { 116 | "name": "stdout", 117 | "output_type": "stream", 118 | "text": [ 119 | "Overwriting requirements.txt\n" 120 | ] 121 | } 122 | ], 123 | "source": [ 124 | "%%writefile requirements.txt\n", 125 | "sqlalchemy==1.4.47\n", 126 | "snowflake-sqlalchemy\n", 127 | "#langchain==0.0.166\n", 128 | "langchain==0.0.190\n", 129 | "sqlalchemy-aurora-data-api\n", 130 | "PyAthena[SQLAlchemy]==2.25.2\n", 131 | "anthropic\n", 132 | "redshift-connector==2.0.910\n", 133 | "sqlalchemy-redshift==0.8.14" 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": 2, 139 | "id": "b55d516c", 140 | "metadata": { 141 | "scrolled": true, 142 | "tags": [] 143 | }, 144 | "outputs": [], 145 | "source": [ 146 | "!pip install -r requirements.txt --quiet" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 3, 152 | "id": "91c153cd", 153 | "metadata": { 154 | "tags": [] 155 | }, 156 | "outputs": [], 157 | "source": [ 158 | "import os\n", 159 | "import json\n", 160 | "import boto3\n", 161 | "\n", 162 | "import sqlalchemy\n", 163 | "from sqlalchemy import create_engine\n", 164 | "from snowflake.sqlalchemy import URL\n", 165 | "\n", 166 | "from langchain.docstore.document import Document\n", 167 | "from langchain import PromptTemplate,SagemakerEndpoint,SQLDatabase, SQLDatabaseChain, LLMChain\n", 168 | "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n", 169 | "from langchain.chains.question_answering import load_qa_chain\n", 170 | "from langchain.prompts.prompt import PromptTemplate\n", 171 | "from langchain.chains import SQLDatabaseSequentialChain\n", 172 | "\n", 173 | "from langchain.chains.api.prompt import API_RESPONSE_PROMPT\n", 174 | "from langchain.chains import APIChain\n", 175 | "from langchain.prompts.prompt import PromptTemplate\n", 176 | "from langchain.chat_models import ChatAnthropic\n", 177 | "from langchain.chains.api import open_meteo_docs\n", 178 | "\n", 179 | "from typing import Dict\n", 180 | "import time" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": 4, 186 | "id": "12108671-0c1b-4b8a-b225-88e1f09049d5", 187 | "metadata": { 188 | "tags": [] 189 | }, 190 | "outputs": [], 191 | "source": [ 192 | "import boto3\n", 193 | "from langchain.llms.bedrock import Bedrock" 194 | ] 195 | }, 196 | { 197 | "cell_type": "markdown", 198 | "id": "074a4144-4053-46c1-ba39-8f64f8fb9e00", 199 | "metadata": {}, 200 | "source": [ 201 | "The data for this COVID-19 dataset is stored in a public accessible S3 bucket. You can use the following command to explore the dataset.\n", 202 | "\n", 203 | "!aws s3 ls s3://covid19-lake/ --no-sign-request" 204 | ] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "id": "87f05601-529a-42d4-8cab-ba0b9445e695", 209 | "metadata": { 210 | "tags": [] 211 | }, 212 | "source": [ 213 | "### One Time Setup\n", 214 | "Some of the resources needed for this notebook such as the IAM policy, AWS Glue database and Glue crawler are created through a cloud formation template. The next block of code does the setup based on user inputs.\n", 215 | "\n", 216 | "**NOTE - The next two blocks of code need to be run only for the first time.**" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "id": "fb560be7-cdea-4ff8-9230-0679252ecf5d", 222 | "metadata": { 223 | "tags": [] 224 | }, 225 | "source": [ 226 | "### User Input\n" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": 6, 232 | "id": "23ab506d-e40d-48ab-af71-82ebb78445c0", 233 | "metadata": { 234 | "tags": [] 235 | }, 236 | "outputs": [], 237 | "source": [ 238 | "#provide user input\n", 239 | "glue_databucket_name = 'genai-mda-110' #Create this bucket in S3\n", 240 | "glue_db_name='genai-workshop120'\n", 241 | "glue_role= 'AWSGlueServiceRole-glueworkshop120'\n", 242 | "glue_crawler_name=glue_db_name+'-crawler120'" 243 | ] 244 | }, 245 | { 246 | "cell_type": "markdown", 247 | "id": "59c5c0b3-d32e-41ff-85fe-dc80586fa737", 248 | "metadata": { 249 | "tags": [] 250 | }, 251 | "source": [ 252 | "### Copy the sample dataset to **your S3 bucket**" 253 | ] 254 | }, 255 | { 256 | "cell_type": "code", 257 | "execution_count": 7, 258 | "id": "ceedd61d-9c21-45b4-b35e-69e5cd047f11", 259 | "metadata": { 260 | "tags": [] 261 | }, 262 | "outputs": [ 263 | { 264 | "name": "stdout", 265 | "output_type": "stream", 266 | "text": [ 267 | "copy: s3://covid19-lake/rearc-covid-19-testing-data/json/states_daily/part-00000-d393c7bd-abc5-403a-b6ed-415bb2259c56-c000.json to s3://genai-mda-110/covid-dataset/part-00000-d393c7bd-abc5-403a-b6ed-415bb2259c56-c000.json\r\n" 268 | ] 269 | } 270 | ], 271 | "source": [ 272 | "!aws s3 cp --recursive s3://covid19-lake/rearc-covid-19-testing-data/json/states_daily/ s3://{glue_databucket_name}/covid-dataset/" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "id": "a6344f3f-40da-4ffb-8b4f-b27c52e1eb02", 278 | "metadata": { 279 | "tags": [] 280 | }, 281 | "source": [ 282 | "### Create IAM Role that runs the crawler" 283 | ] 284 | }, 285 | { 286 | "cell_type": "code", 287 | "execution_count": 8, 288 | "id": "2d905b91", 289 | "metadata": {}, 290 | "outputs": [ 291 | { 292 | "name": "stdout", 293 | "output_type": "stream", 294 | "text": [ 295 | "AWS Account Number: 715253196401\n", 296 | "AWS Region: us-east-1\n" 297 | ] 298 | } 299 | ], 300 | "source": [ 301 | "import boto3\n", 302 | "import os\n", 303 | "# Retrieve the AWS account number\n", 304 | "sts_client = boto3.client('sts')\n", 305 | "account_number = sts_client.get_caller_identity().get('Account')\n", 306 | "# Retrieve the AWS region\n", 307 | "#region = os.environ['AWS_REGION']\n", 308 | "region = boto3.session.Session().region_name\n", 309 | "print(\"AWS Account Number:\", account_number)\n", 310 | "print(\"AWS Region:\", region)\n", 311 | "trust_policy=\"\"\"{\n", 312 | " \"Version\": \"2012-10-17\",\n", 313 | " \"Statement\": [\n", 314 | " {\n", 315 | " \"Sid\": \"\",\n", 316 | " \"Effect\": \"Allow\",\n", 317 | " \"Principal\": {\n", 318 | " \"Service\": \"glue.amazonaws.com\"\n", 319 | " },\n", 320 | " \"Action\": \"sts:AssumeRole\"\n", 321 | " }\n", 322 | " ]\n", 323 | "}\"\"\"\n", 324 | "managed_policy=\"\"\"{\n", 325 | " \"Version\": \"2012-10-17\",\n", 326 | " \"Statement\": [\n", 327 | " {\n", 328 | " \"Action\": [\n", 329 | " \"glue:*\"\n", 330 | " ],\n", 331 | " \"Resource\": [\n", 332 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":catalog\",\n", 333 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":database/*\",\n", 334 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":table/*\"\n", 335 | " ],\n", 336 | " \"Effect\": \"Allow\",\n", 337 | " \"Sid\": \"Readcrawlerresources\"\n", 338 | " },\n", 339 | " {\n", 340 | " \"Action\": [\n", 341 | " \"logs:CreateLogGroup\",\n", 342 | " \"logs:CreateLogStream\",\n", 343 | " \"logs:PutLogEvents\"\n", 344 | " ],\n", 345 | " \"Resource\": [\n", 346 | " \"arn:aws:glue:\"\"\"+region+\"\"\":\"\"\"+account_number+\"\"\":log-group:/aws-glue/crawlers*\",\n", 347 | " \"arn:aws:logs:*:*:/aws-glue/*\",\n", 348 | " \"arn:aws:logs:*:*:/customlogs/*\"\n", 349 | " ],\n", 350 | " \"Effect\": \"Allow\",\n", 351 | " \"Sid\": \"ReadlogResources\"\n", 352 | " },\n", 353 | " {\n", 354 | " \"Action\": [\n", 355 | " \"s3:PutObject\",\n", 356 | " \"s3:GetObject\",\n", 357 | " \"s3:PutBucketLogging\",\n", 358 | " \"s3:ListBucket\",\n", 359 | " \"s3:PutBucketVersioning\"\n", 360 | " ],\n", 361 | " \"Resource\": [\n", 362 | " \"arn:aws:s3:::\"\"\"+glue_databucket_name+\"\"\"\",\n", 363 | " \"arn:aws:s3:::\"\"\"+glue_databucket_name+\"\"\"/*\"\n", 364 | " ],\n", 365 | " \"Effect\": \"Allow\",\n", 366 | " \"Sid\": \"ReadS3Resources\"\n", 367 | " }\n", 368 | " ]\n", 369 | " }\"\"\"\n", 370 | "print(managed_policy, file=open('managed-policy.json', 'w'))\n", 371 | "print(trust_policy, file=open('trust-policy.json', 'w'))" 372 | ] 373 | }, 374 | { 375 | "cell_type": "code", 376 | "execution_count": 9, 377 | "id": "c4c27763-e1e4-4b41-bfa4-7ff8f6500c31", 378 | "metadata": { 379 | "tags": [] 380 | }, 381 | "outputs": [ 382 | { 383 | "name": "stdout", 384 | "output_type": "stream", 385 | "text": [ 386 | "AWSGlueServiceRole-glueworkshop120\n", 387 | "managed-policy-AWSGlueServiceRole-glueworkshop120\n", 388 | "{\n", 389 | " \"Role\": {\n", 390 | " \"Path\": \"/\",\n", 391 | " \"RoleName\": \"AWSGlueServiceRole-glueworkshop120\",\n", 392 | " \"RoleId\": \"AROA2NCDJUZY5EUJL3TRS\",\n", 393 | " \"Arn\": \"arn:aws:iam::715253196401:role/AWSGlueServiceRole-glueworkshop120\",\n", 394 | " \"CreateDate\": \"2023-06-06T19:20:31Z\",\n", 395 | " \"AssumeRolePolicyDocument\": {\n", 396 | " \"Version\": \"2012-10-17\",\n", 397 | " \"Statement\": [\n", 398 | " {\n", 399 | " \"Sid\": \"\",\n", 400 | " \"Effect\": \"Allow\",\n", 401 | " \"Principal\": {\n", 402 | " \"Service\": \"glue.amazonaws.com\"\n", 403 | " },\n", 404 | " \"Action\": \"sts:AssumeRole\"\n", 405 | " }\n", 406 | " ]\n", 407 | " }\n", 408 | " }\n", 409 | "}\n", 410 | "arn:aws:iam::715253196401:policy/managed-policy-AWSGlueServiceRole-glueworkshop120\n" 411 | ] 412 | } 413 | ], 414 | "source": [ 415 | "%%sh -s \"$glue_role\" \n", 416 | "echo $1 \n", 417 | "glue_role=\"$1\"\n", 418 | "managed_policy_name=\"managed-policy-$glue_role\"\n", 419 | "echo $managed_policy_name\n", 420 | "aws iam create-role --role-name $glue_role --assume-role-policy-document file://trust-policy.json\n", 421 | "output=$(aws iam create-policy --policy-document file://managed-policy.json --policy-name $managed_policy_name)\n", 422 | "arn=$(echo \"$output\" | grep -oP '\"Arn\": \"\\K[^\"]+')\n", 423 | "echo \"$arn\"\n", 424 | "aws iam attach-role-policy --policy-arn $arn --role-name $glue_role" 425 | ] 426 | }, 427 | { 428 | "cell_type": "code", 429 | "execution_count": null, 430 | "id": "6fe28c2a-60c4-400a-9693-395e304c5164", 431 | "metadata": { 432 | "tags": [] 433 | }, 434 | "outputs": [ 435 | { 436 | "name": "stdout", 437 | "output_type": "stream", 438 | "text": [ 439 | "Successfully created database\n", 440 | "Successfully created crawler\n", 441 | "Successfully started crawler. The crawler may take 2-5 mins to detect the schema.\n", 442 | "Crawler 'genai-workshop120-crawler120' status: RUNNING\n", 443 | "Crawler 'genai-workshop120-crawler120' status: RUNNING\n", 444 | "Crawler 'genai-workshop120-crawler120' status: RUNNING\n", 445 | "Crawler 'genai-workshop120-crawler120' status: RUNNING\n", 446 | "Crawler 'genai-workshop120-crawler120' status: RUNNING\n", 447 | "Crawler 'genai-workshop120-crawler120' status: RUNNING\n", 448 | "Crawler 'genai-workshop120-crawler120' status: STOPPING\n", 449 | "Crawler 'genai-workshop120-crawler120' status: STOPPING\n", 450 | "Crawler 'genai-workshop120-crawler120' status: STOPPING\n", 451 | "Crawler 'genai-workshop120-crawler120' status: STOPPING\n", 452 | "Crawler 'genai-workshop120-crawler120' status: STOPPING\n", 453 | "Crawler 'genai-workshop120-crawler120' status: STOPPING\n", 454 | "Crawler 'genai-workshop120-crawler120' status: STOPPING\n", 455 | "Crawler 'genai-workshop120-crawler120' status: READY\n" 456 | ] 457 | } 458 | ], 459 | "source": [ 460 | "import boto3\n", 461 | "\n", 462 | "client = boto3.client('glue')\n", 463 | "\n", 464 | "# Create database \n", 465 | "try:\n", 466 | " response = client.create_database(\n", 467 | " DatabaseInput={\n", 468 | " 'Name': glue_db_name,\n", 469 | " 'Description': 'This database is created using Python boto3',\n", 470 | " }\n", 471 | " )\n", 472 | " print(\"Successfully created database\")\n", 473 | "except:\n", 474 | " print(\"error in creating database. Check if the database already exists\")\n", 475 | "\n", 476 | "#introducing some lag for the iam role to create\n", 477 | "time.sleep(20) \n", 478 | "\n", 479 | "# Create Glue Crawler \n", 480 | "try:\n", 481 | "\n", 482 | " response = client.create_crawler(\n", 483 | " Name=glue_crawler_name,\n", 484 | " Role=glue_role,\n", 485 | " DatabaseName=glue_db_name,\n", 486 | " Targets={\n", 487 | " 'S3Targets': [\n", 488 | " {\n", 489 | " 'Path': 's3://{BUCKET_NAME}/covid-dataset/'.format(BUCKET_NAME =glue_databucket_name)\n", 490 | " }\n", 491 | " ]\n", 492 | " },\n", 493 | " TablePrefix=''\n", 494 | " )\n", 495 | " \n", 496 | " print(\"Successfully created crawler\")\n", 497 | "except:\n", 498 | " print(\"error in creating crawler. However, if the crawler already exists, the crawler will run.\")\n", 499 | "\n", 500 | "# Run the Crawler\n", 501 | "try:\n", 502 | " response = client.start_crawler(Name=glue_crawler_name )\n", 503 | " print(\"Successfully started crawler. The crawler may take 2-5 mins to detect the schema.\")\n", 504 | " while True:\n", 505 | " # Get the crawler status\n", 506 | " response = client.get_crawler(Name=glue_crawler_name)\n", 507 | " # Extract the crawler state\n", 508 | " status = response['Crawler']['State']\n", 509 | " # Print the crawler status\n", 510 | " print(f\"Crawler '{glue_crawler_name}' status: {status}\")\n", 511 | " if status == 'READY': # Replace 'READY' with the desired completed state\n", 512 | " break # Exit the loop if the desired state is reached\n", 513 | "\n", 514 | " time.sleep(10) # Sleep for 10 seconds before checking the status again\n", 515 | " \n", 516 | "except:\n", 517 | " print(\"error in starting crawler. Check the logs for the error details.\")" 518 | ] 519 | }, 520 | { 521 | "cell_type": "markdown", 522 | "id": "4132ffc3-6947-49b6-b627-fae3df870b88", 523 | "metadata": { 524 | "tags": [] 525 | }, 526 | "source": [ 527 | "Before proceeding to the next step, check the status of the crawler. It should change from RUNNING to READY. " 528 | ] 529 | }, 530 | { 531 | "cell_type": "markdown", 532 | "id": "b51d1d0e-33fb-46ca-b82f-6294ea867cae", 533 | "metadata": { 534 | "tags": [] 535 | }, 536 | "source": [ 537 | "### Step 1 - Connect to databases using SQL Alchemy. \n", 538 | "\n", 539 | "Under the hood, LangChain uses SQLAlchemy to connect to SQL databases. The SQLDatabaseChain can therefore be used with any SQL dialect supported by SQLAlchemy, \n", 540 | "such as MS SQL, MySQL, MariaDB, PostgreSQL, Oracle SQL, and SQLite. Please refer to the SQLAlchemy documentation for more information about requirements for connecting to your database. \n" 541 | ] 542 | }, 543 | { 544 | "cell_type": "markdown", 545 | "id": "e5f5ce28-9b33-4061-8655-2b297d5c24a2", 546 | "metadata": { 547 | "tags": [] 548 | }, 549 | "source": [ 550 | "**Important**: The code below establishes a database connection for data sources and Large Language Models. Please note that the solution will only work if the database connection for your sources is defined in the cell below. Please refer to the Pre-requisites section. If your use case requires data from Aurora MySQL alone, then please comment out other data sources. Furthermore, please update the cluster details and variables for Aurora MySQL accordingly." 551 | ] 552 | }, 553 | { 554 | "cell_type": "code", 555 | "execution_count": 11, 556 | "id": "1583cade", 557 | "metadata": { 558 | "scrolled": true, 559 | "tags": [] 560 | }, 561 | "outputs": [], 562 | "source": [ 563 | "#define connections\n", 564 | "\n", 565 | "# collect credentials from Secrets Manager\n", 566 | "#Refer here on how to use AWS Secrets Manager - https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html\n", 567 | "# client = boto3.client('secretsmanager') region = client.meta.region_name\n", 568 | "region=os.getenv(\"AWS_REGION\")\n", 569 | "\n", 570 | "#LLM \n", 571 | "# Updated the Bedrock client initiation because of the library issue\n", 572 | "BEDROCK_CLIENT = boto3.client(\"bedrock-runtime\", 'us-east-1')\n", 573 | "inference_modifier = {'max_tokens_to_sample':350,\n", 574 | " \"temperature\":0.0,\n", 575 | " \"top_k\":50,\n", 576 | " }\n", 577 | "llm = Bedrock(model_id='anthropic.claude-v2', client=BEDROCK_CLIENT, model_kwargs = inference_modifier,)\n", 578 | "\n", 579 | "#S3\n", 580 | "# connect to s3 using athena\n", 581 | "## athena variables\n", 582 | "connathena=f\"athena.{region}.amazonaws.com\" \n", 583 | "portathena='443' #Update, if port is different\n", 584 | "schemaathena=glue_db_name #from user defined params\n", 585 | "s3stagingathena=f's3://{glue_databucket_name}/athenaresults/'#from cfn params\n", 586 | "wkgrpathena='primary'#Update, if workgroup is different\n", 587 | "# tablesathena=['dataset']#[]\n", 588 | "## Create the athena connection string\n", 589 | "connection_string = f\"awsathena+rest://@{connathena}:{portathena}/{schemaathena}?s3_staging_dir={s3stagingathena}/&work_group={wkgrpathena}\"\n", 590 | "## Create the athena SQLAlchemy engine\n", 591 | "engine_athena = create_engine(connection_string, echo=False)\n", 592 | "dbathena = SQLDatabase(engine_athena)\n", 593 | "\n", 594 | "gdc = [schemaathena] \n" 595 | ] 596 | }, 597 | { 598 | "cell_type": "code", 599 | "execution_count": 12, 600 | "id": "c5a4f470-cbc5-44c0-a348-c49792ab2229", 601 | "metadata": { 602 | "tags": [] 603 | }, 604 | "outputs": [ 605 | { 606 | "data": { 607 | "text/plain": [ 608 | "['genai-workshop120']" 609 | ] 610 | }, 611 | "execution_count": 12, 612 | "metadata": {}, 613 | "output_type": "execute_result" 614 | } 615 | ], 616 | "source": [ 617 | "gdc" 618 | ] 619 | }, 620 | { 621 | "cell_type": "markdown", 622 | "id": "1ea21757-b08a-438b-a5a7-79d85a9a9085", 623 | "metadata": {}, 624 | "source": [ 625 | "### Step 2 - Generate Dynamic Prompt Templates\n", 626 | "Build a consolidated view of Glue Data Catalog by combining metadata stored for all the databases in pipe delimited format." 627 | ] 628 | }, 629 | { 630 | "cell_type": "code", 631 | "execution_count": 13, 632 | "id": "08a3373d-9285-4fab-81b5-51e5364590b5", 633 | "metadata": { 634 | "scrolled": true, 635 | "tags": [] 636 | }, 637 | "outputs": [ 638 | { 639 | "name": "stdout", 640 | "output_type": "stream", 641 | "text": [ 642 | "s3|genai-workshop120|covid_dataset|totaltestresults\n", 643 | "s3|genai-workshop120|covid_dataset|fips\n", 644 | "s3|genai-workshop120|covid_dataset|deathincrease\n", 645 | "s3|genai-workshop120|covid_dataset|hospitalizedincrease\n", 646 | "s3|genai-workshop120|covid_dataset|negativeincrease\n", 647 | "s3|genai-workshop120|covid_dataset|positiveincrease\n", 648 | "s3|genai-workshop120|covid_dataset|totaltestresultsincrease\n", 649 | "s3|genai-workshop120|covid_dataset|negative\n", 650 | "s3|genai-workshop120|covid_dataset|pending\n", 651 | "api|meteo|weather|weather\n" 652 | ] 653 | } 654 | ], 655 | "source": [ 656 | "#Generate Dynamic prompts to populate the Glue Data Catalog\n", 657 | "#harvest aws crawler metadata\n", 658 | "\n", 659 | "def parse_catalog():\n", 660 | " #Connect to Glue catalog\n", 661 | " #get metadata of redshift serverless tables\n", 662 | " columns_str=''\n", 663 | " \n", 664 | " #define glue cient\n", 665 | " glue_client = boto3.client('glue')\n", 666 | " \n", 667 | " for db in gdc:\n", 668 | " response = glue_client.get_tables(DatabaseName =db)\n", 669 | " for tables in response['TableList']:\n", 670 | " #classification in the response for s3 and other databases is different. Set classification based on the response location\n", 671 | " if tables['StorageDescriptor']['Location'].startswith('s3'): classification='s3' \n", 672 | " else: classification = tables['Parameters']['classification']\n", 673 | " for columns in tables['StorageDescriptor']['Columns']:\n", 674 | " dbname,tblname,colname=tables['DatabaseName'],tables['Name'],columns['Name']\n", 675 | " columns_str=columns_str+f'\\n{classification}|{dbname}|{tblname}|{colname}' \n", 676 | " #API\n", 677 | " ## Append the metadata of the API to the unified glue data catalog\n", 678 | " columns_str=columns_str+'\\n'+('api|meteo|weather|weather')\n", 679 | " return columns_str\n", 680 | "\n", 681 | "glue_catalog = parse_catalog()\n", 682 | "\n", 683 | "#display a few lines from the catalog\n", 684 | "print('\\n'.join(glue_catalog.splitlines()[-10:]) )\n" 685 | ] 686 | }, 687 | { 688 | "cell_type": "markdown", 689 | "id": "a94e6770-42c3-402b-a60e-9c21fb99d5f6", 690 | "metadata": { 691 | "tags": [] 692 | }, 693 | "source": [ 694 | "### Step 3 - Define Functions to 1/ determine the best data channel to answer the user query, 2/ Generate response to user query" 695 | ] 696 | }, 697 | { 698 | "cell_type": "markdown", 699 | "id": "adda3714-3f32-4480-9526-91cca37489d1", 700 | "metadata": {}, 701 | "source": [ 702 | "In this code sample, we use the Anthropic Model to generate inferences. You can utilize SageMaker JumpStart models to achieve the same. \n", 703 | "Guidance on how to use the JumpStart Models is available in the notebook - mda_with_llm_langchain_smjumpstart_flant5xl" 704 | ] 705 | }, 706 | { 707 | "cell_type": "code", 708 | "execution_count": 14, 709 | "id": "4efcc59b", 710 | "metadata": { 711 | "tags": [] 712 | }, 713 | "outputs": [], 714 | "source": [ 715 | "#Function 1 'Infer Channel'\n", 716 | "#define a function that infers the channel/database/table and sets the database for querying\n", 717 | "def identify_channel(query):\n", 718 | " #Prompt 1 'Infer Channel'\n", 719 | " ##set prompt template. It instructs the llm on how to evaluate and respond to the llm. It is referred to as dynamic since glue data catalog is first getting generated and appended to the prompt.\n", 720 | " prompt_template = \"\"\"\\n\\nHuman:\n", 721 | " From the table below, find the database (in column database) which will contain the data (in corresponding column_names) to answer the question \n", 722 | " {query} \\n\n", 723 | " \"\"\"+glue_catalog +\"\"\" \n", 724 | " Give your answer as database == \n", 725 | " Also,give your answer as database.table == \\n\\nAssistant:\n", 726 | " \"\"\"\n", 727 | " ##define prompt 1\n", 728 | " PROMPT_channel = PromptTemplate( template=prompt_template, input_variables=[\"query\"] )\n", 729 | "\n", 730 | " # define llm chain\n", 731 | " llm_chain = LLMChain(prompt=PROMPT_channel, llm=llm)\n", 732 | " #run the query and save to generated texts\n", 733 | " generated_texts = llm_chain.run(query)\n", 734 | " print(generated_texts)\n", 735 | "\n", 736 | " #set the channel from where the query can be answered\n", 737 | " if 's3' in generated_texts: \n", 738 | " channel='db'\n", 739 | " db=dbathena\n", 740 | " print(\"SET database to athena\")\n", 741 | " elif 'api' in generated_texts: \n", 742 | " channel='api'\n", 743 | " print(\"SET database to weather api\") \n", 744 | " else: raise Exception(\"User question cannot be answered by any of the channels mentioned in the catalog\")\n", 745 | " print(\"Step complete. Channel is: \", channel)\n", 746 | " \n", 747 | " return channel, db\n", 748 | "\n", 749 | "#Function 2 'Run Query'\n", 750 | "#define a function that infers the channel/database/table and sets the database for querying\n", 751 | "def run_query(query):\n", 752 | "\n", 753 | " channel, db = identify_channel(query) #call the identify channel function first\n", 754 | "\n", 755 | " ##Prompt 2 'Run Query'\n", 756 | " #after determining the data channel, run the Langchain SQL Database chain to convert 'text to sql' and run the query against the source data channel. \n", 757 | " #provide rules for running the SQL queries in default template--> table info.\n", 758 | "\n", 759 | " _DEFAULT_TEMPLATE = \"\"\"\\n\\nHuman: Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.\n", 760 | "\n", 761 | " Do not append 'Query:' to SQLQuery.\n", 762 | " \n", 763 | " Display SQLResult after the query is run in plain english that users can understand. \n", 764 | "\n", 765 | " Provide answer in simple english statement.\n", 766 | " \n", 767 | " Only use the following tables:\n", 768 | "\n", 769 | " {table_info}\n", 770 | " If someone asks for the sales, they really mean the tickit.sales table.\n", 771 | " If someone asks for the sales date, they really mean the column tickit.sales.saletime.\n", 772 | " Just provide the SQL query. Do not provide any preamble or additional text.\n", 773 | " Question: {input}\\n\\nAssistant: Here is the SQL Query: \"\"\"\n", 774 | "\n", 775 | " PROMPT_sql = PromptTemplate(\n", 776 | " input_variables=[\"input\", \"table_info\", \"dialect\"], template=_DEFAULT_TEMPLATE\n", 777 | " )\n", 778 | "\n", 779 | " \n", 780 | " if channel=='db':\n", 781 | " db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT_sql, verbose=True, return_intermediate_steps=False)\n", 782 | " response=db_chain.run(query)\n", 783 | " elif channel=='api':\n", 784 | " chain_api = APIChain.from_llm_and_api_docs(llm, open_meteo_docs.OPEN_METEO_DOCS, verbose=True)\n", 785 | " response=chain_api.run(query)\n", 786 | " else: raise Exception(\"Unlisted channel. Check your unified catalog\")\n", 787 | " return response\n", 788 | "\n" 789 | ] 790 | }, 791 | { 792 | "cell_type": "markdown", 793 | "id": "390a92cd-e1b4-4feb-ab7a-f97030ba7f84", 794 | "metadata": {}, 795 | "source": [ 796 | "### Step 4 - Run the run_query function that in turn calls the Langchain SQL Database chain to convert 'text to sql' and runs the query against the source data channel\n", 797 | "\n", 798 | "Some samples are provided below for test runs. Uncomment the query to run." 799 | ] 800 | }, 801 | { 802 | "cell_type": "code", 803 | "execution_count": 15, 804 | "id": "f82599a2", 805 | "metadata": { 806 | "tags": [] 807 | }, 808 | "outputs": [ 809 | { 810 | "name": "stdout", 811 | "output_type": "stream", 812 | "text": [ 813 | "\n", 814 | "database == s3\n", 815 | "database.table == s3|genai-workshop120|covid_dataset\n", 816 | "\n", 817 | " From the selected database, the column_names required to answer the given question are:\n", 818 | "\n", 819 | "column_names = [state,\n", 820 | "SET database to athena\n", 821 | "Step complete. Channel is: db\n", 822 | "\n", 823 | "\n", 824 | "\u001b[1m> Entering new SQLDatabaseChain chain...\u001b[0m\n", 825 | "Which States reported the least and maximum deaths?\n", 826 | "SQLQuery:\u001b[32;1m\u001b[1;3mSELECT state, MIN(death) AS least_deaths, MAX(death) AS most_deaths \n", 827 | "FROM covid_dataset \n", 828 | "GROUP BY state\n", 829 | "ORDER BY least_deaths, most_deaths;\u001b[0m\n", 830 | "SQLResult: \u001b[33;1m\u001b[1;3m[('AS', 0.0, 0.0), ('MP', 0.0, 2.0), ('AK', 0.0, 305.0), ('WY', 0.0, 682.0), ('DC', 0.0, 1030.0), ('DE', 0.0, 1473.0), ('ND', 0.0, 1478.0), ('ID', 0.0, 1879.0), ('UT', 0.0, 1976.0), ('NE', 0.0, 2113.0), ('WV', 0.0, 2325.0), ('NV', 0.0, 5037.0), ('AR', 0.0, 5417.0), ('MO', 0.0, 8161.0), ('AL', 0.0, 10149.0), ('NC', 0.0, 11502.0), ('IN', 0.0, 12737.0), ('AZ', 0.0, 16328.0), ('MI', 0.0, 16658.0), ('NJ', 0.0, 23574.0), ('VI', 1.0, 25.0), ('GU', 1.0, 133.0), ('HI', 1.0, 445.0), ('ME', 1.0, 706.0), ('NH', 1.0, 1184.0), ('MT', 1.0, 1381.0), ('SD', 1.0, 1900.0), ('PR', 1.0, 2059.0), ('NM', 1.0, 3808.0), ('OK', 1.0, 4534.0), ('KS', 1.0, 4816.0), ('KY', 1.0, 4819.0), ('IA', 1.0, 5558.0), ('CO', 1.0, 5989.0), ('MN', 1.0, 6550.0), ('MS', 1.0, 6808.0), ('CT', 1.0, 7704.0), ('SC', 1.0, 8754.0), ('VA', 1.0, 9596.0), ('OH', 1.0, 17656.0), ('GA', 1.0, 17906.0), ('IL', 1.0, 23014.0), ('PA', 1.0, 24349.0), ('TX', 1.0, 44451.0), ('VT', 2.0, 208.0), ('WA', 2.0, 5041.0), ('MD', 2.0, 7955.0), ('LA', 2.0, 9748.0), ('TN', 2.0, 11543.0), ('MA', 2.0, 16417.0), ('FL', 2.0, 32266.0), ('OR', 3.0, 2296.0), ('RI', 3.0, 2541.0), ('WI', 3.0, 7106.0), ('NY', 3.0, 39029.0), ('CA', 4.0, 54124.0)]\u001b[0m\n", 831 | "Answer:\u001b[32;1m\u001b[1;3mThe states that reported the least deaths are American Samoa and the Northern Mariana Islands with 0 deaths.\n", 832 | "The states that reported the maximum deaths are California with 54,124 deaths and New York with 39,029 deaths.\u001b[0m\n", 833 | "\u001b[1m> Finished chain.\u001b[0m\n", 834 | "----------------------------------------------------------------------\n", 835 | "SQL and response from user query Which States reported the least and maximum deaths? \n", 836 | " The states that reported the least deaths are American Samoa and the Northern Mariana Islands with 0 deaths.\n", 837 | "The states that reported the maximum deaths are California with 54,124 deaths and New York with 39,029 deaths.\n" 838 | ] 839 | } 840 | ], 841 | "source": [ 842 | "# Enter the query\n", 843 | "## Few queries to try out - \n", 844 | "#athena - Healthcare - Covid dataset\n", 845 | "# query = \"\"\"How many covid hospitalizations were reported in NY in June of 2021?\"\"\" \n", 846 | "query = \"\"\"Which States reported the least and maximum deaths?\"\"\" \n", 847 | "\n", 848 | "#api - product - weather\n", 849 | "# query = \"\"\"What is the weather like right now in New York City in degrees Farenheit?\"\"\"\n", 850 | "\n", 851 | "#Response from Langchain\n", 852 | "response = run_query(query)\n", 853 | "print(\"----------------------------------------------------------------------\")\n", 854 | "print(f'SQL and response from user query {query} \\n {response}')" 855 | ] 856 | }, 857 | { 858 | "cell_type": "markdown", 859 | "id": "69371bdc-537f-4e5e-a004-99d852097862", 860 | "metadata": {}, 861 | "source": [ 862 | "### Clean-up\n", 863 | "After you run the modern data architecture with Generative AI, make sure to clean up any resources that won’t be utilized. Delete the data in Amazon S3 and make sure to stop any SageMaker Studio notebook instances to not incur any further charges.\n" 864 | ] 865 | } 866 | ], 867 | "metadata": { 868 | "availableInstances": [ 869 | { 870 | "_defaultOrder": 0, 871 | "_isFastLaunch": true, 872 | "category": "General purpose", 873 | "gpuNum": 0, 874 | "hideHardwareSpecs": false, 875 | "memoryGiB": 4, 876 | "name": "ml.t3.medium", 877 | "vcpuNum": 2 878 | }, 879 | { 880 | "_defaultOrder": 1, 881 | "_isFastLaunch": false, 882 | "category": "General purpose", 883 | "gpuNum": 0, 884 | "hideHardwareSpecs": false, 885 | "memoryGiB": 8, 886 | "name": "ml.t3.large", 887 | "vcpuNum": 2 888 | }, 889 | { 890 | "_defaultOrder": 2, 891 | "_isFastLaunch": false, 892 | "category": "General purpose", 893 | "gpuNum": 0, 894 | "hideHardwareSpecs": false, 895 | "memoryGiB": 16, 896 | "name": "ml.t3.xlarge", 897 | "vcpuNum": 4 898 | }, 899 | { 900 | "_defaultOrder": 3, 901 | "_isFastLaunch": false, 902 | "category": "General purpose", 903 | "gpuNum": 0, 904 | "hideHardwareSpecs": false, 905 | "memoryGiB": 32, 906 | "name": "ml.t3.2xlarge", 907 | "vcpuNum": 8 908 | }, 909 | { 910 | "_defaultOrder": 4, 911 | "_isFastLaunch": true, 912 | "category": "General purpose", 913 | "gpuNum": 0, 914 | "hideHardwareSpecs": false, 915 | "memoryGiB": 8, 916 | "name": "ml.m5.large", 917 | "vcpuNum": 2 918 | }, 919 | { 920 | "_defaultOrder": 5, 921 | "_isFastLaunch": false, 922 | "category": "General purpose", 923 | "gpuNum": 0, 924 | "hideHardwareSpecs": false, 925 | "memoryGiB": 16, 926 | "name": "ml.m5.xlarge", 927 | "vcpuNum": 4 928 | }, 929 | { 930 | "_defaultOrder": 6, 931 | "_isFastLaunch": false, 932 | "category": "General purpose", 933 | "gpuNum": 0, 934 | "hideHardwareSpecs": false, 935 | "memoryGiB": 32, 936 | "name": "ml.m5.2xlarge", 937 | "vcpuNum": 8 938 | }, 939 | { 940 | "_defaultOrder": 7, 941 | "_isFastLaunch": false, 942 | "category": "General purpose", 943 | "gpuNum": 0, 944 | "hideHardwareSpecs": false, 945 | "memoryGiB": 64, 946 | "name": "ml.m5.4xlarge", 947 | "vcpuNum": 16 948 | }, 949 | { 950 | "_defaultOrder": 8, 951 | "_isFastLaunch": false, 952 | "category": "General purpose", 953 | "gpuNum": 0, 954 | "hideHardwareSpecs": false, 955 | "memoryGiB": 128, 956 | "name": "ml.m5.8xlarge", 957 | "vcpuNum": 32 958 | }, 959 | { 960 | "_defaultOrder": 9, 961 | "_isFastLaunch": false, 962 | "category": "General purpose", 963 | "gpuNum": 0, 964 | "hideHardwareSpecs": false, 965 | "memoryGiB": 192, 966 | "name": "ml.m5.12xlarge", 967 | "vcpuNum": 48 968 | }, 969 | { 970 | "_defaultOrder": 10, 971 | "_isFastLaunch": false, 972 | "category": "General purpose", 973 | "gpuNum": 0, 974 | "hideHardwareSpecs": false, 975 | "memoryGiB": 256, 976 | "name": "ml.m5.16xlarge", 977 | "vcpuNum": 64 978 | }, 979 | { 980 | "_defaultOrder": 11, 981 | "_isFastLaunch": false, 982 | "category": "General purpose", 983 | "gpuNum": 0, 984 | "hideHardwareSpecs": false, 985 | "memoryGiB": 384, 986 | "name": "ml.m5.24xlarge", 987 | "vcpuNum": 96 988 | }, 989 | { 990 | "_defaultOrder": 12, 991 | "_isFastLaunch": false, 992 | "category": "General purpose", 993 | "gpuNum": 0, 994 | "hideHardwareSpecs": false, 995 | "memoryGiB": 8, 996 | "name": "ml.m5d.large", 997 | "vcpuNum": 2 998 | }, 999 | { 1000 | "_defaultOrder": 13, 1001 | "_isFastLaunch": false, 1002 | "category": "General purpose", 1003 | "gpuNum": 0, 1004 | "hideHardwareSpecs": false, 1005 | "memoryGiB": 16, 1006 | "name": "ml.m5d.xlarge", 1007 | "vcpuNum": 4 1008 | }, 1009 | { 1010 | "_defaultOrder": 14, 1011 | "_isFastLaunch": false, 1012 | "category": "General purpose", 1013 | "gpuNum": 0, 1014 | "hideHardwareSpecs": false, 1015 | "memoryGiB": 32, 1016 | "name": "ml.m5d.2xlarge", 1017 | "vcpuNum": 8 1018 | }, 1019 | { 1020 | "_defaultOrder": 15, 1021 | "_isFastLaunch": false, 1022 | "category": "General purpose", 1023 | "gpuNum": 0, 1024 | "hideHardwareSpecs": false, 1025 | "memoryGiB": 64, 1026 | "name": "ml.m5d.4xlarge", 1027 | "vcpuNum": 16 1028 | }, 1029 | { 1030 | "_defaultOrder": 16, 1031 | "_isFastLaunch": false, 1032 | "category": "General purpose", 1033 | "gpuNum": 0, 1034 | "hideHardwareSpecs": false, 1035 | "memoryGiB": 128, 1036 | "name": "ml.m5d.8xlarge", 1037 | "vcpuNum": 32 1038 | }, 1039 | { 1040 | "_defaultOrder": 17, 1041 | "_isFastLaunch": false, 1042 | "category": "General purpose", 1043 | "gpuNum": 0, 1044 | "hideHardwareSpecs": false, 1045 | "memoryGiB": 192, 1046 | "name": "ml.m5d.12xlarge", 1047 | "vcpuNum": 48 1048 | }, 1049 | { 1050 | "_defaultOrder": 18, 1051 | "_isFastLaunch": false, 1052 | "category": "General purpose", 1053 | "gpuNum": 0, 1054 | "hideHardwareSpecs": false, 1055 | "memoryGiB": 256, 1056 | "name": "ml.m5d.16xlarge", 1057 | "vcpuNum": 64 1058 | }, 1059 | { 1060 | "_defaultOrder": 19, 1061 | "_isFastLaunch": false, 1062 | "category": "General purpose", 1063 | "gpuNum": 0, 1064 | "hideHardwareSpecs": false, 1065 | "memoryGiB": 384, 1066 | "name": "ml.m5d.24xlarge", 1067 | "vcpuNum": 96 1068 | }, 1069 | { 1070 | "_defaultOrder": 20, 1071 | "_isFastLaunch": false, 1072 | "category": "General purpose", 1073 | "gpuNum": 0, 1074 | "hideHardwareSpecs": true, 1075 | "memoryGiB": 0, 1076 | "name": "ml.geospatial.interactive", 1077 | "supportedImageNames": [ 1078 | "sagemaker-geospatial-v1-0" 1079 | ], 1080 | "vcpuNum": 0 1081 | }, 1082 | { 1083 | "_defaultOrder": 21, 1084 | "_isFastLaunch": true, 1085 | "category": "Compute optimized", 1086 | "gpuNum": 0, 1087 | "hideHardwareSpecs": false, 1088 | "memoryGiB": 4, 1089 | "name": "ml.c5.large", 1090 | "vcpuNum": 2 1091 | }, 1092 | { 1093 | "_defaultOrder": 22, 1094 | "_isFastLaunch": false, 1095 | "category": "Compute optimized", 1096 | "gpuNum": 0, 1097 | "hideHardwareSpecs": false, 1098 | "memoryGiB": 8, 1099 | "name": "ml.c5.xlarge", 1100 | "vcpuNum": 4 1101 | }, 1102 | { 1103 | "_defaultOrder": 23, 1104 | "_isFastLaunch": false, 1105 | "category": "Compute optimized", 1106 | "gpuNum": 0, 1107 | "hideHardwareSpecs": false, 1108 | "memoryGiB": 16, 1109 | "name": "ml.c5.2xlarge", 1110 | "vcpuNum": 8 1111 | }, 1112 | { 1113 | "_defaultOrder": 24, 1114 | "_isFastLaunch": false, 1115 | "category": "Compute optimized", 1116 | "gpuNum": 0, 1117 | "hideHardwareSpecs": false, 1118 | "memoryGiB": 32, 1119 | "name": "ml.c5.4xlarge", 1120 | "vcpuNum": 16 1121 | }, 1122 | { 1123 | "_defaultOrder": 25, 1124 | "_isFastLaunch": false, 1125 | "category": "Compute optimized", 1126 | "gpuNum": 0, 1127 | "hideHardwareSpecs": false, 1128 | "memoryGiB": 72, 1129 | "name": "ml.c5.9xlarge", 1130 | "vcpuNum": 36 1131 | }, 1132 | { 1133 | "_defaultOrder": 26, 1134 | "_isFastLaunch": false, 1135 | "category": "Compute optimized", 1136 | "gpuNum": 0, 1137 | "hideHardwareSpecs": false, 1138 | "memoryGiB": 96, 1139 | "name": "ml.c5.12xlarge", 1140 | "vcpuNum": 48 1141 | }, 1142 | { 1143 | "_defaultOrder": 27, 1144 | "_isFastLaunch": false, 1145 | "category": "Compute optimized", 1146 | "gpuNum": 0, 1147 | "hideHardwareSpecs": false, 1148 | "memoryGiB": 144, 1149 | "name": "ml.c5.18xlarge", 1150 | "vcpuNum": 72 1151 | }, 1152 | { 1153 | "_defaultOrder": 28, 1154 | "_isFastLaunch": false, 1155 | "category": "Compute optimized", 1156 | "gpuNum": 0, 1157 | "hideHardwareSpecs": false, 1158 | "memoryGiB": 192, 1159 | "name": "ml.c5.24xlarge", 1160 | "vcpuNum": 96 1161 | }, 1162 | { 1163 | "_defaultOrder": 29, 1164 | "_isFastLaunch": true, 1165 | "category": "Accelerated computing", 1166 | "gpuNum": 1, 1167 | "hideHardwareSpecs": false, 1168 | "memoryGiB": 16, 1169 | "name": "ml.g4dn.xlarge", 1170 | "vcpuNum": 4 1171 | }, 1172 | { 1173 | "_defaultOrder": 30, 1174 | "_isFastLaunch": false, 1175 | "category": "Accelerated computing", 1176 | "gpuNum": 1, 1177 | "hideHardwareSpecs": false, 1178 | "memoryGiB": 32, 1179 | "name": "ml.g4dn.2xlarge", 1180 | "vcpuNum": 8 1181 | }, 1182 | { 1183 | "_defaultOrder": 31, 1184 | "_isFastLaunch": false, 1185 | "category": "Accelerated computing", 1186 | "gpuNum": 1, 1187 | "hideHardwareSpecs": false, 1188 | "memoryGiB": 64, 1189 | "name": "ml.g4dn.4xlarge", 1190 | "vcpuNum": 16 1191 | }, 1192 | { 1193 | "_defaultOrder": 32, 1194 | "_isFastLaunch": false, 1195 | "category": "Accelerated computing", 1196 | "gpuNum": 1, 1197 | "hideHardwareSpecs": false, 1198 | "memoryGiB": 128, 1199 | "name": "ml.g4dn.8xlarge", 1200 | "vcpuNum": 32 1201 | }, 1202 | { 1203 | "_defaultOrder": 33, 1204 | "_isFastLaunch": false, 1205 | "category": "Accelerated computing", 1206 | "gpuNum": 4, 1207 | "hideHardwareSpecs": false, 1208 | "memoryGiB": 192, 1209 | "name": "ml.g4dn.12xlarge", 1210 | "vcpuNum": 48 1211 | }, 1212 | { 1213 | "_defaultOrder": 34, 1214 | "_isFastLaunch": false, 1215 | "category": "Accelerated computing", 1216 | "gpuNum": 1, 1217 | "hideHardwareSpecs": false, 1218 | "memoryGiB": 256, 1219 | "name": "ml.g4dn.16xlarge", 1220 | "vcpuNum": 64 1221 | }, 1222 | { 1223 | "_defaultOrder": 35, 1224 | "_isFastLaunch": false, 1225 | "category": "Accelerated computing", 1226 | "gpuNum": 1, 1227 | "hideHardwareSpecs": false, 1228 | "memoryGiB": 61, 1229 | "name": "ml.p3.2xlarge", 1230 | "vcpuNum": 8 1231 | }, 1232 | { 1233 | "_defaultOrder": 36, 1234 | "_isFastLaunch": false, 1235 | "category": "Accelerated computing", 1236 | "gpuNum": 4, 1237 | "hideHardwareSpecs": false, 1238 | "memoryGiB": 244, 1239 | "name": "ml.p3.8xlarge", 1240 | "vcpuNum": 32 1241 | }, 1242 | { 1243 | "_defaultOrder": 37, 1244 | "_isFastLaunch": false, 1245 | "category": "Accelerated computing", 1246 | "gpuNum": 8, 1247 | "hideHardwareSpecs": false, 1248 | "memoryGiB": 488, 1249 | "name": "ml.p3.16xlarge", 1250 | "vcpuNum": 64 1251 | }, 1252 | { 1253 | "_defaultOrder": 38, 1254 | "_isFastLaunch": false, 1255 | "category": "Accelerated computing", 1256 | "gpuNum": 8, 1257 | "hideHardwareSpecs": false, 1258 | "memoryGiB": 768, 1259 | "name": "ml.p3dn.24xlarge", 1260 | "vcpuNum": 96 1261 | }, 1262 | { 1263 | "_defaultOrder": 39, 1264 | "_isFastLaunch": false, 1265 | "category": "Memory Optimized", 1266 | "gpuNum": 0, 1267 | "hideHardwareSpecs": false, 1268 | "memoryGiB": 16, 1269 | "name": "ml.r5.large", 1270 | "vcpuNum": 2 1271 | }, 1272 | { 1273 | "_defaultOrder": 40, 1274 | "_isFastLaunch": false, 1275 | "category": "Memory Optimized", 1276 | "gpuNum": 0, 1277 | "hideHardwareSpecs": false, 1278 | "memoryGiB": 32, 1279 | "name": "ml.r5.xlarge", 1280 | "vcpuNum": 4 1281 | }, 1282 | { 1283 | "_defaultOrder": 41, 1284 | "_isFastLaunch": false, 1285 | "category": "Memory Optimized", 1286 | "gpuNum": 0, 1287 | "hideHardwareSpecs": false, 1288 | "memoryGiB": 64, 1289 | "name": "ml.r5.2xlarge", 1290 | "vcpuNum": 8 1291 | }, 1292 | { 1293 | "_defaultOrder": 42, 1294 | "_isFastLaunch": false, 1295 | "category": "Memory Optimized", 1296 | "gpuNum": 0, 1297 | "hideHardwareSpecs": false, 1298 | "memoryGiB": 128, 1299 | "name": "ml.r5.4xlarge", 1300 | "vcpuNum": 16 1301 | }, 1302 | { 1303 | "_defaultOrder": 43, 1304 | "_isFastLaunch": false, 1305 | "category": "Memory Optimized", 1306 | "gpuNum": 0, 1307 | "hideHardwareSpecs": false, 1308 | "memoryGiB": 256, 1309 | "name": "ml.r5.8xlarge", 1310 | "vcpuNum": 32 1311 | }, 1312 | { 1313 | "_defaultOrder": 44, 1314 | "_isFastLaunch": false, 1315 | "category": "Memory Optimized", 1316 | "gpuNum": 0, 1317 | "hideHardwareSpecs": false, 1318 | "memoryGiB": 384, 1319 | "name": "ml.r5.12xlarge", 1320 | "vcpuNum": 48 1321 | }, 1322 | { 1323 | "_defaultOrder": 45, 1324 | "_isFastLaunch": false, 1325 | "category": "Memory Optimized", 1326 | "gpuNum": 0, 1327 | "hideHardwareSpecs": false, 1328 | "memoryGiB": 512, 1329 | "name": "ml.r5.16xlarge", 1330 | "vcpuNum": 64 1331 | }, 1332 | { 1333 | "_defaultOrder": 46, 1334 | "_isFastLaunch": false, 1335 | "category": "Memory Optimized", 1336 | "gpuNum": 0, 1337 | "hideHardwareSpecs": false, 1338 | "memoryGiB": 768, 1339 | "name": "ml.r5.24xlarge", 1340 | "vcpuNum": 96 1341 | }, 1342 | { 1343 | "_defaultOrder": 47, 1344 | "_isFastLaunch": false, 1345 | "category": "Accelerated computing", 1346 | "gpuNum": 1, 1347 | "hideHardwareSpecs": false, 1348 | "memoryGiB": 16, 1349 | "name": "ml.g5.xlarge", 1350 | "vcpuNum": 4 1351 | }, 1352 | { 1353 | "_defaultOrder": 48, 1354 | "_isFastLaunch": false, 1355 | "category": "Accelerated computing", 1356 | "gpuNum": 1, 1357 | "hideHardwareSpecs": false, 1358 | "memoryGiB": 32, 1359 | "name": "ml.g5.2xlarge", 1360 | "vcpuNum": 8 1361 | }, 1362 | { 1363 | "_defaultOrder": 49, 1364 | "_isFastLaunch": false, 1365 | "category": "Accelerated computing", 1366 | "gpuNum": 1, 1367 | "hideHardwareSpecs": false, 1368 | "memoryGiB": 64, 1369 | "name": "ml.g5.4xlarge", 1370 | "vcpuNum": 16 1371 | }, 1372 | { 1373 | "_defaultOrder": 50, 1374 | "_isFastLaunch": false, 1375 | "category": "Accelerated computing", 1376 | "gpuNum": 1, 1377 | "hideHardwareSpecs": false, 1378 | "memoryGiB": 128, 1379 | "name": "ml.g5.8xlarge", 1380 | "vcpuNum": 32 1381 | }, 1382 | { 1383 | "_defaultOrder": 51, 1384 | "_isFastLaunch": false, 1385 | "category": "Accelerated computing", 1386 | "gpuNum": 1, 1387 | "hideHardwareSpecs": false, 1388 | "memoryGiB": 256, 1389 | "name": "ml.g5.16xlarge", 1390 | "vcpuNum": 64 1391 | }, 1392 | { 1393 | "_defaultOrder": 52, 1394 | "_isFastLaunch": false, 1395 | "category": "Accelerated computing", 1396 | "gpuNum": 4, 1397 | "hideHardwareSpecs": false, 1398 | "memoryGiB": 192, 1399 | "name": "ml.g5.12xlarge", 1400 | "vcpuNum": 48 1401 | }, 1402 | { 1403 | "_defaultOrder": 53, 1404 | "_isFastLaunch": false, 1405 | "category": "Accelerated computing", 1406 | "gpuNum": 4, 1407 | "hideHardwareSpecs": false, 1408 | "memoryGiB": 384, 1409 | "name": "ml.g5.24xlarge", 1410 | "vcpuNum": 96 1411 | }, 1412 | { 1413 | "_defaultOrder": 54, 1414 | "_isFastLaunch": false, 1415 | "category": "Accelerated computing", 1416 | "gpuNum": 8, 1417 | "hideHardwareSpecs": false, 1418 | "memoryGiB": 768, 1419 | "name": "ml.g5.48xlarge", 1420 | "vcpuNum": 192 1421 | }, 1422 | { 1423 | "_defaultOrder": 55, 1424 | "_isFastLaunch": false, 1425 | "category": "Accelerated computing", 1426 | "gpuNum": 8, 1427 | "hideHardwareSpecs": false, 1428 | "memoryGiB": 1152, 1429 | "name": "ml.p4d.24xlarge", 1430 | "vcpuNum": 96 1431 | }, 1432 | { 1433 | "_defaultOrder": 56, 1434 | "_isFastLaunch": false, 1435 | "category": "Accelerated computing", 1436 | "gpuNum": 8, 1437 | "hideHardwareSpecs": false, 1438 | "memoryGiB": 1152, 1439 | "name": "ml.p4de.24xlarge", 1440 | "vcpuNum": 96 1441 | } 1442 | ], 1443 | "instance_type": "ml.t3.medium", 1444 | "kernelspec": { 1445 | "display_name": "conda_python3", 1446 | "language": "python", 1447 | "name": "conda_python3" 1448 | }, 1449 | "language_info": { 1450 | "codemirror_mode": { 1451 | "name": "ipython", 1452 | "version": 3 1453 | }, 1454 | "file_extension": ".py", 1455 | "mimetype": "text/x-python", 1456 | "name": "python", 1457 | "nbconvert_exporter": "python", 1458 | "pygments_lexer": "ipython3", 1459 | "version": "3.10.8" 1460 | } 1461 | }, 1462 | "nbformat": 4, 1463 | "nbformat_minor": 5 1464 | } 1465 | --------------------------------------------------------------------------------