├── .github └── ISSUE_TEMPLATE │ └── bug_report.md ├── .gitignore ├── 01_Text_generation ├── 01_text_and_code_generation_w_bedrock.ipynb └── README.md ├── 02_Knowledge_Bases_and_RAG ├── 1_create-kb-and-ingest-documents.ipynb ├── 2_managed-rag-with-retrieve-and-generate-api.ipynb ├── 3_customized-rag-with-retrieve-api.ipynb ├── 4_clean-up.ipynb ├── README.md ├── images │ ├── bkb_illustration.png │ ├── data_ingestion.png │ ├── retrieve_and_generate_api.png │ └── retrieve_api.png ├── requirements.txt └── utility.py ├── 03_Model_customization ├── README.md ├── bedrock-models-fine-tuning │ ├── README.md │ ├── amazon-nova │ │ ├── 01_fine-tune_Amazon_Nova.ipynb │ │ ├── 02_Inference_Amazon_Nova.ipynb │ │ ├── README.md │ │ ├── finetune-diagram.png │ │ └── requirements.txt │ ├── claude-haiku │ │ ├── 01_setup.ipynb │ │ └── 02_fine-tune_Claude_Haiku.ipynb │ └── meta-llama │ │ └── Llama-3.2 Multi-modal cusotmization │ │ ├── 01_setup_llama-3.2.ipynb │ │ ├── 02_fine-tune_llama3.2.ipynb │ │ └── README.md └── continued Pre-training │ ├── 01_setup.ipynb │ ├── 02_continued_pretraining_titan_text.ipynb │ └── 03_cleanup.ipynb ├── 04_Image_and_Multimodal ├── 1_titan-multimodal-embeddings-lab.ipynb ├── 2_nova-canvas-lab.ipynb ├── 3_nova-reel-lab.ipynb ├── README.md ├── images │ ├── 71-txt-2-img.png │ ├── 72-img-2-img.png │ ├── generated_dog.png │ ├── mask.png │ ├── multimodal-embeddings.png │ ├── octank_color_palette.JPG │ ├── sd.png │ ├── sketch_dog.png │ ├── three_pots-add_mask.png │ ├── three_pots-add_mask_INVERTED.png │ ├── three_pots-center_pot_mask.png │ ├── three_pots-center_pot_mask_INVERTED.png │ ├── three_pots.jpg │ ├── titan_image_generator_playground.png │ ├── video_input.png │ └── woman-in-gallery.jpg └── requirements.txt ├── 05_Agents ├── 01_create_agent.ipynb ├── 02_associate_knowledge_base_to_agent.ipynb ├── 03_invoke_agent.ipynb ├── 04_clean_up_agent_resources.ipynb ├── 05_bonus_inline_agents.ipynb ├── README.md ├── agent.py ├── images │ ├── agent-architecture.png │ ├── agent-architecture_1.png │ ├── agent-orchestration-flow-pink.png │ └── agent_components.png ├── kb_documents │ ├── Restaurant_Childrens_Menu.pdf │ ├── Restaurant_Dinner_Menu.pdf │ └── Restaurant_week_specials.pdf ├── knowledge_base.py └── requirements.txt ├── 06_OpenSource_examples ├── README.md ├── advance-langgraph-multi-agent-setup.ipynb ├── data │ ├── section_doc_store.pkl │ ├── section_vector_store.pkl │ ├── synthetic_travel_data.csv │ └── travel_guides │ │ ├── asheville_travel_guide.pdf │ │ ├── atlanta_travel_guide.pdf │ │ ├── austin_travel_guide.pdf │ │ ├── baltimore_travel_guide.pdf │ │ ├── boston_travel_guide.pdf │ │ ├── charleston_travel_guide.pdf │ │ ├── charlotte_travel_guide.pdf │ │ ├── chicago_travel_guide.pdf │ │ ├── cincinnati_travel_guide.pdf │ │ ├── dallas_travel_guide.pdf │ │ ├── denver_travel_guide.pdf │ │ ├── jacksonville_travel_guide.pdf │ │ └── kansas_city_travel_guide.pdf ├── find-relevant-information-using-RAG.ipynb ├── images │ ├── Hotel_booking_confirmation_light.png │ ├── RAG-travellers.png │ ├── ReACT_agents.png │ ├── ReWoO_agents.png │ ├── agent_action_thought.png │ ├── agent_alignment_chart.png │ ├── agents_itinerary.png │ ├── agents_memory_light.png │ ├── amsterdam.jpeg │ ├── chatbot_bedrock.png │ ├── converse_api.png │ ├── graph_state_light.png │ ├── human-in-the-loop.png │ ├── multi-agent-travel-bot.png │ ├── multi_memory_light.png │ ├── rag-architecture.png │ ├── rag-workflow.png │ ├── react_tool_call.png │ ├── short-vs-long.png │ ├── travel_agent_light.png │ ├── vegas.png │ └── weather.jpg ├── intermediate-langgraph-agent-setup-w-tools.ipynb ├── ragas-agent-evaluation.ipynb ├── requirements.txt ├── simple-crewai-agent-setup.ipynb ├── simple-langgraph-agent-setup.ipynb ├── text-generation-with-langchain │ ├── 00_simple_chatbot.ipynb │ ├── 01_zero_shot_generation.ipynb │ ├── 02_code_interpret_w_langchain.ipynb │ ├── 03_code_translate_w_langchain.ipynb │ ├── 04_long_text_summarization_using_LCEL_chains_on_langchain.ipynb │ ├── images │ │ ├── bedrock_langchain.jpg │ │ ├── code-interpret-langchain.png │ │ └── code-translation-langchain.png │ ├── letters │ │ └── 2022-letter.txt │ └── requirements.txt └── utils.py ├── 07_Cross_Region_Inference ├── Getting_started_with_Cross-region_Inference.ipynb └── README.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── RELEASE_NOTES.md └── imgs ├── 10-overview.png ├── 11-overview.png ├── 41-text-simple-1.png ├── 42-text-summarization-2.png ├── 51-simple-rag.png ├── 52-rag-with-external-data.png ├── 53-rag-with-pinecone.png ├── Chatbot_lang.png ├── Embeddings_lang (1).png ├── Embeddings_pinecone_lang.png ├── bedrock-code-gen.png ├── chatbot_bedrock.png ├── chatbot_lang.png ├── context-aware-chatbot.png ├── embeddings_lang.png └── logic-overview.png /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: Amazon Bedrock Workshop - [Bug] 5 | labels: bug 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. This could be anything from: 12 | 13 | 1. A typo or text changes that make the lab better.... to 14 | 2. minor annoying code that still works.. to 15 | 3. breaking changes 16 | 17 | **To Reproduce** 18 | Steps to reproduce the behavior (or some version of this): 19 | 1. Go to '...' 20 | 2. Click on '....' 21 | 3. Scroll down to '....' 22 | 4. See error 23 | 24 | **Expected behavior** 25 | A clear and concise description of what you expected to happen. 26 | 27 | **Screenshots** 28 | If applicable, add screenshots to help explain your problem. 29 | 30 | **Desktop (please complete the following information):** 31 | - OS: [e.g. iOS] 32 | - Browser [e.g. chrome, safari] 33 | - Version [e.g. 22] 34 | 35 | **Smartphone (please complete the following information):** 36 | - Device: [e.g. iPhone6] 37 | - OS: [e.g. iOS8.1] 38 | - Browser [e.g. stock browser, safari] 39 | - Version [e.g. 22] 40 | 41 | **! Additional context !** 42 | - Sagemaker Studio Kernel used 43 | - Instance used 44 | - commit link to the workshop if you're not using the latest 45 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # OS and temporary files: 2 | __pycache__ 3 | .ipynb_checkpoints 4 | .venv 5 | .DS_Store 6 | .idea 7 | 8 | # Files generated by the workshop: 9 | /dependencies 10 | data/ 11 | 12 | .idea 13 | AWSCLIV2.pkg 14 | -------------------------------------------------------------------------------- /01_Text_generation/README.md: -------------------------------------------------------------------------------- 1 | # Amazon Bedrock Workshop - Module 1 - Text Generation 2 | 3 | In this workshop you'll learn the basics of the Amazon Bedrock Invoke API, explore the more powerful Converse API with its features like multi-turn conversation, streaming, and function calling, apply these APIs across various foundation models, and compare results across different state-of-the-art models. 4 | 5 | Please go through the module by executing the notebook labs in their naming order. 6 | 7 | ## Contributing 8 | 9 | We welcome community contributions! Please ensure your sample aligns with [AWS best practices](https://aws.amazon.com/architecture/well-architected/), and please update the **Contents** section of this README file with a link to your sample, along with a description. -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/2_managed-rag-with-retrieve-and-generate-api.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Module 2 - Build RAG-powered Q&A Application with **RetrieveAndGenerate API**\n", 8 | "\n", 9 | "----\n", 10 | "\n", 11 | "This notebook provides sample code and step-by-step instructions for building a fully-managed question-answering (Q&A) application using a **RetrieveAndGenerate API** of Amazon Bedrock Knowledge Bases.\n", 12 | "\n", 13 | "----\n", 14 | "\n", 15 | "### Introduction\n", 16 | "\n", 17 | "In the previous notebook, we demonstrated how to create a Knowledge Base in Amazon Bedrock — including setting up an S3 data source, configuring an Amazon OpenSearch Serverless (AOSS) vector index, and ingesting documents for retrieval-augmented generation (RAG).\n", 18 | "\n", 19 | "In this notebook, we take the next step: building a Q&A application that can query that Knowledge Base using the `RetrieveAndGenerate` API. This API allows you to retrieve the most relevant content from your Knowledge Base based on a user’s query and automatically pass that information to a foundation model (FM) to generate a grounded, context-aware response.\n", 20 | "\n", 21 | "This is a classic example of the RAG pattern — where external data is dynamically retrieved at query time and incorporated into the model’s prompt to improve relevance, accuracy, and transparency. In this solution, retrieved knowledge base content comes with source attribution, helping end users understand the origin of the response and minimizing the risk of model hallucinations.\n", 22 | "\n", 23 | "![BKB illustration](./images/retrieve_and_generate_api.png)\n", 24 | "\n", 25 | "### Pre-requisites\n", 26 | "\n", 27 | "In order to run this notebook, you should have successfully completed the previous notebook lab:\n", 28 | "- [1_create-kb-and-ingest-documents.ipynb](./1\\_create-kb-and-ingest-documents.ipynb).\n", 29 | "\n", 30 | "Also, please make sure that you have enabled the following model access in _Amazon Bedrock Console_:\n", 31 | "- `Amazon Nova Micro`\n", 32 | "- `Amazon Titan Text Embeddings V2`\n" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "## 1. Setup\n", 40 | "\n", 41 | "### 1.1 Import the required libraries" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": null, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "# Standard library imports\n", 51 | "import os\n", 52 | "import sys\n", 53 | "import json\n", 54 | "import time\n", 55 | "\n", 56 | "# Third-party imports\n", 57 | "import boto3\n", 58 | "from botocore.client import Config\n", 59 | "from botocore.exceptions import ClientError\n", 60 | "\n", 61 | "# Local imports\n", 62 | "import utility\n", 63 | "\n", 64 | "# Print SDK versions\n", 65 | "print(f\"Python version: {sys.version.split()[0]}\")\n", 66 | "print(f\"Boto3 SDK version: {boto3.__version__}\")" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | "### 1.2 Initial setup for clients and global variables" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": {}, 80 | "outputs": [], 81 | "source": [ 82 | "%store -r bedrock_kb_id" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "# Create boto3 session and set AWS region\n", 92 | "boto_session = boto3.Session()\n", 93 | "aws_region = boto_session.region_name\n", 94 | "\n", 95 | "# Create boto3 clients for Bedrock\n", 96 | "bedrock_config = Config(connect_timeout=120, read_timeout=120, retries={'max_attempts': 0})\n", 97 | "bedrock_agent_client = boto3.client('bedrock-agent-runtime', config=bedrock_config)\n", 98 | "\n", 99 | "# Set the Bedrock model to use for text generation\n", 100 | "model_id = 'amazon.nova-micro-v1:0'\n", 101 | "model_arn = f'arn:aws:bedrock:{aws_region}::foundation-model/{model_id}'\n", 102 | "\n", 103 | "# Print configurations\n", 104 | "print(\"AWS Region:\", aws_region)\n", 105 | "print(\"Bedrock Knowledge Base ID:\", bedrock_kb_id)" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "## 2. Fully-managed RAG with **RetreiveAndGenerate API**\n", 113 | "\n", 114 | "The `RetrieveAndGenerate` API provides a fully managed way to implement the Retrieval-Augmented Generation (RAG) pattern with Amazon Bedrock Knowledge Bases.\n", 115 | "\n", 116 | "When a user submits a query, the API automatically converts the query into vector embeddings, performs a similarity search against the Knowledge Base, and retrieves the most relevant document chunks. These search results are then injected into the foundation model's prompt as additional context, enabling the model to generate more accurate and grounded responses.\n", 117 | "\n", 118 | "For multi-turn conversations, Knowledge Bases also maintain short-term conversational memory — allowing the API to return more contextually relevant answers across a dialogue.\n", 119 | "\n", 120 | "The output of the `RetrieveAndGenerate` API includes:\n", 121 | "\n", 122 | "- The **generated response** from the foundation model\n", 123 | "\n", 124 | "- **Source attribution** metadata for the retrieved content\n", 125 | "\n", 126 | "- The **actual retrieved text chunks** from the Knowledge Base\n", 127 | "\n", 128 | "This makes it easy to build RAG-powered applications with trusted, explainable answers — without having to manage retrieval pipelines or prompt construction yourself.\n", 129 | "\n", 130 | "### 2.1 Retrieve and Generate Example\n", 131 | "\n", 132 | "Let’s now see the `RetrieveAndGenerate` API in action and showcase a fully managed RAG workflow in Amazon Bedrock.\n", 133 | "\n", 134 | "In this example, we’ll use the Knowledge Base built in the previous lab — containing Amazon Shareholder Letters — to demonstrate how the API retrieves relevant information and generates a grounded response to a user query." 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": null, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "user_query = \"How does Amazon use technology to better serve its customers?\"\n", 144 | "\n", 145 | "response = bedrock_agent_client.retrieve_and_generate(\n", 146 | " input={\n", 147 | " 'text': user_query\n", 148 | " },\n", 149 | " retrieveAndGenerateConfiguration={\n", 150 | " 'type': 'KNOWLEDGE_BASE',\n", 151 | " 'knowledgeBaseConfiguration': {\n", 152 | " 'knowledgeBaseId': bedrock_kb_id,\n", 153 | " 'modelArn': model_arn\n", 154 | " }\n", 155 | " }\n", 156 | ")\n", 157 | "\n", 158 | "print(\"Final reply:\\n\", response['output']['text'])" 159 | ] 160 | }, 161 | { 162 | "cell_type": "markdown", 163 | "metadata": {}, 164 | "source": [ 165 | "### 2.2 Understanding Citations\n", 166 | "\n", 167 | "Citations play a critical role in retrieval-augmented generation (RAG) systems by helping users verify the accuracy of a response and providing transparency into the source of information. Let's now look at the `citations` past of the Knowledge Base response:" 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": null, 173 | "metadata": {}, 174 | "outputs": [], 175 | "source": [ 176 | "print(\"Citations:\\n\", json.dumps(response[\"citations\"], indent=2, default=str))" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": {}, 182 | "source": [ 183 | "Here, the response includes a `generatedResponsePart` field, which contains the natural language answer generated by the model. Each `generatedResponsePart` is paired with `retrievedReferences`, which lists the specific pieces of content from the knowledge base that were used to ground that part of the response. These references include the original source text (`content.text`), as well as metadata like the source URI and page number, so users can easily trace information back to its original document. This structure ensures that answers are both helpful and verifiable, allowing users to explore the source material directly and build trust in the response." 184 | ] 185 | }, 186 | { 187 | "cell_type": "markdown", 188 | "metadata": {}, 189 | "source": [ 190 | "## 3. Conclusions and Next Steps\n", 191 | "\n", 192 | "In this notebook, we built a fully-managed RAG-powered Q&A application using the `RetrieveAndGenerate` API from Amazon Bedrock Knowledge Bases.\n", 193 | "\n", 194 | "We demonstrated how this API simplifies the RAG workflow by automatically retrieving relevant content from a knowledge base and generating grounded, context-aware responses using a foundation model. The responses also include source references, allowing users to easily trace answers back to the original documents.\n", 195 | "\n", 196 | "This approach enables you to quickly build reliable, transparent Q&A solutions without managing the complexity of prompt engineering or retrieval logic manually.\n", 197 | "\n", 198 | "### Next Steps\n", 199 | "\n", 200 | "If you are looking for more flexibility and control over your RAG workflow, Amazon Bedrock Knowledge Bases also provides a `Retrieve` API. This API allows you to perform semantic and/or keyword search over your knowledge base and retrieve the most relevant document chunks, which you can then use to build custom prompts or workflows tailored to your application needs.\n", 201 | "\n", 202 | "To explore this approach, check out the next notebook:\n", 203 | "\n", 204 | "  **NEXT ▶** [3_customized-rag-with-retrieve-api](./3\\_customized-rag-with-retrieve-api.ipynb)" 205 | ] 206 | } 207 | ], 208 | "metadata": { 209 | "kernelspec": { 210 | "display_name": "Python 3 (ipykernel)", 211 | "language": "python", 212 | "name": "python3" 213 | }, 214 | "language_info": { 215 | "codemirror_mode": { 216 | "name": "ipython", 217 | "version": 3 218 | }, 219 | "file_extension": ".py", 220 | "mimetype": "text/x-python", 221 | "name": "python", 222 | "nbconvert_exporter": "python", 223 | "pygments_lexer": "ipython3", 224 | "version": "3.11.11" 225 | } 226 | }, 227 | "nbformat": 4, 228 | "nbformat_minor": 4 229 | } 230 | -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/README.md: -------------------------------------------------------------------------------- 1 | # Amazon Bedrock Workshop - Module 2 - Knowledge Bases and RAG 2 | 3 | This workshop module provides an introductory demostration of Amazon Bedrock Knowledge Bases capabilities. 4 | 5 | Please go through the module by executing the notebook labs in their naming order. 6 | 7 | ## Contributing 8 | 9 | We welcome community contributions! Please ensure your sample aligns with [AWS best practices](https://aws.amazon.com/architecture/well-architected/), and please update the **Contents** section of this README file with a link to your sample, along with a description. -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/images/bkb_illustration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/02_Knowledge_Bases_and_RAG/images/bkb_illustration.png -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/images/data_ingestion.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/02_Knowledge_Bases_and_RAG/images/data_ingestion.png -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/images/retrieve_and_generate_api.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/02_Knowledge_Bases_and_RAG/images/retrieve_and_generate_api.png -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/images/retrieve_api.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/02_Knowledge_Bases_and_RAG/images/retrieve_api.png -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/requirements.txt: -------------------------------------------------------------------------------- 1 | awscli 2 | boto3 3 | botocore 4 | opensearch-py 5 | requests-aws4auth -------------------------------------------------------------------------------- /02_Knowledge_Bases_and_RAG/utility.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import random 3 | import time 4 | import json 5 | 6 | suffix = random.randrange(200, 900) 7 | boto3_session = boto3.session.Session() 8 | region_name = boto3_session.region_name 9 | iam_client = boto3_session.client('iam') 10 | account_number = boto3.client('sts').get_caller_identity().get('Account') 11 | identity = boto3.client('sts').get_caller_identity()['Arn'] 12 | 13 | encryption_policy_name = f"bedrock-sample-rag-sp-{suffix}" 14 | network_policy_name = f"bedrock-sample-rag-np-{suffix}" 15 | access_policy_name = f'bedrock-sample-rag-ap-{suffix}' 16 | bedrock_execution_role_name = f'AmazonBedrockExecutionRoleForKnowledgeBase_{suffix}' 17 | fm_policy_name = f'AmazonBedrockFoundationModelPolicyForKnowledgeBase_{suffix}' 18 | s3_policy_name = f'AmazonBedrockS3PolicyForKnowledgeBase_{suffix}' 19 | sm_policy_name = f'AmazonBedrockSecretPolicyForKnowledgeBase_{suffix}' 20 | oss_policy_name = f'AmazonBedrockOSSPolicyForKnowledgeBase_{suffix}' 21 | 22 | sm_policy_flag = False 23 | 24 | def create_bedrock_execution_role(bucket_name): 25 | foundation_model_policy_document = { 26 | "Version": "2012-10-17", 27 | "Statement": [ 28 | { 29 | "Effect": "Allow", 30 | "Action": [ 31 | "bedrock:InvokeModel", 32 | ], 33 | "Resource": [ 34 | f"arn:aws:bedrock:{region_name}::foundation-model/amazon.titan-embed-text-v1", 35 | f"arn:aws:bedrock:{region_name}::foundation-model/amazon.titan-embed-text-v2:0" 36 | ] 37 | } 38 | ] 39 | } 40 | 41 | s3_policy_document = { 42 | "Version": "2012-10-17", 43 | "Statement": [ 44 | { 45 | "Effect": "Allow", 46 | "Action": [ 47 | "s3:GetObject", 48 | "s3:ListBucket" 49 | ], 50 | "Resource": [ 51 | f"arn:aws:s3:::{bucket_name}", 52 | f"arn:aws:s3:::{bucket_name}/*" 53 | ], 54 | "Condition": { 55 | "StringEquals": { 56 | "aws:ResourceAccount": f"{account_number}" 57 | } 58 | } 59 | } 60 | ] 61 | } 62 | 63 | assume_role_policy_document = { 64 | "Version": "2012-10-17", 65 | "Statement": [ 66 | { 67 | "Effect": "Allow", 68 | "Principal": { 69 | "Service": "bedrock.amazonaws.com" 70 | }, 71 | "Action": "sts:AssumeRole" 72 | } 73 | ] 74 | } 75 | # create policies based on the policy documents 76 | fm_policy = iam_client.create_policy( 77 | PolicyName=fm_policy_name, 78 | PolicyDocument=json.dumps(foundation_model_policy_document), 79 | Description='Policy for accessing foundation model', 80 | ) 81 | 82 | s3_policy = iam_client.create_policy( 83 | PolicyName=s3_policy_name, 84 | PolicyDocument=json.dumps(s3_policy_document), 85 | Description='Policy for reading documents from s3') 86 | 87 | # create bedrock execution role 88 | bedrock_kb_execution_role = iam_client.create_role( 89 | RoleName=bedrock_execution_role_name, 90 | AssumeRolePolicyDocument=json.dumps(assume_role_policy_document), 91 | Description='Amazon Bedrock Knowledge Base Execution Role for accessing OSS and S3', 92 | MaxSessionDuration=3600 93 | ) 94 | 95 | # fetch arn of the policies and role created above 96 | bedrock_kb_execution_role_arn = bedrock_kb_execution_role['Role']['Arn'] 97 | s3_policy_arn = s3_policy["Policy"]["Arn"] 98 | fm_policy_arn = fm_policy["Policy"]["Arn"] 99 | 100 | 101 | # attach policies to Amazon Bedrock execution role 102 | iam_client.attach_role_policy( 103 | RoleName=bedrock_kb_execution_role["Role"]["RoleName"], 104 | PolicyArn=fm_policy_arn 105 | ) 106 | iam_client.attach_role_policy( 107 | RoleName=bedrock_kb_execution_role["Role"]["RoleName"], 108 | PolicyArn=s3_policy_arn 109 | ) 110 | return bedrock_kb_execution_role 111 | 112 | 113 | def create_oss_policy_attach_bedrock_execution_role(collection_id, bedrock_kb_execution_role): 114 | # define oss policy document 115 | oss_policy_document = { 116 | "Version": "2012-10-17", 117 | "Statement": [ 118 | { 119 | "Effect": "Allow", 120 | "Action": [ 121 | "aoss:APIAccessAll" 122 | ], 123 | "Resource": [ 124 | f"arn:aws:aoss:{region_name}:{account_number}:collection/{collection_id}" 125 | ] 126 | } 127 | ] 128 | } 129 | oss_policy = iam_client.create_policy( 130 | PolicyName=oss_policy_name, 131 | PolicyDocument=json.dumps(oss_policy_document), 132 | Description='Policy for accessing opensearch serverless', 133 | ) 134 | oss_policy_arn = oss_policy["Policy"]["Arn"] 135 | 136 | iam_client.attach_role_policy( 137 | RoleName=bedrock_kb_execution_role["Role"]["RoleName"], 138 | PolicyArn=oss_policy_arn 139 | ) 140 | return oss_policy_arn 141 | 142 | 143 | def create_policies_in_oss(vector_store_name, aoss_client, bedrock_kb_execution_role_arn): 144 | encryption_policy = aoss_client.create_security_policy( 145 | name=encryption_policy_name, 146 | policy=json.dumps( 147 | { 148 | 'Rules': [{'Resource': ['collection/' + vector_store_name], 149 | 'ResourceType': 'collection'}], 150 | 'AWSOwnedKey': True 151 | }), 152 | type='encryption' 153 | ) 154 | 155 | network_policy = aoss_client.create_security_policy( 156 | name=network_policy_name, 157 | policy=json.dumps( 158 | [ 159 | {'Rules': [{'Resource': ['collection/' + vector_store_name], 160 | 'ResourceType': 'collection'}], 161 | 'AllowFromPublic': True} 162 | ]), 163 | type='network' 164 | ) 165 | access_policy = aoss_client.create_access_policy( 166 | name=access_policy_name, 167 | policy=json.dumps( 168 | [ 169 | { 170 | 'Rules': [ 171 | { 172 | 'Resource': ['collection/' + vector_store_name], 173 | 'Permission': [ 174 | 'aoss:CreateCollectionItems', 175 | 'aoss:DeleteCollectionItems', 176 | 'aoss:UpdateCollectionItems', 177 | 'aoss:DescribeCollectionItems'], 178 | 'ResourceType': 'collection' 179 | }, 180 | { 181 | 'Resource': ['index/' + vector_store_name + '/*'], 182 | 'Permission': [ 183 | 'aoss:CreateIndex', 184 | 'aoss:DeleteIndex', 185 | 'aoss:UpdateIndex', 186 | 'aoss:DescribeIndex', 187 | 'aoss:ReadDocument', 188 | 'aoss:WriteDocument'], 189 | 'ResourceType': 'index' 190 | }], 191 | 'Principal': [identity, bedrock_kb_execution_role_arn], 192 | 'Description': 'Easy data policy'} 193 | ]), 194 | type='data' 195 | ) 196 | return encryption_policy, network_policy, access_policy 197 | 198 | 199 | def delete_iam_role_and_policies(): 200 | fm_policy_arn = f"arn:aws:iam::{account_number}:policy/{fm_policy_name}" 201 | s3_policy_arn = f"arn:aws:iam::{account_number}:policy/{s3_policy_name}" 202 | oss_policy_arn = f"arn:aws:iam::{account_number}:policy/{oss_policy_name}" 203 | sm_policy_arn = f"arn:aws:iam::{account_number}:policy/{sm_policy_name}" 204 | 205 | iam_client.detach_role_policy( 206 | RoleName=bedrock_execution_role_name, 207 | PolicyArn=s3_policy_arn 208 | ) 209 | iam_client.detach_role_policy( 210 | RoleName=bedrock_execution_role_name, 211 | PolicyArn=fm_policy_arn 212 | ) 213 | iam_client.detach_role_policy( 214 | RoleName=bedrock_execution_role_name, 215 | PolicyArn=oss_policy_arn 216 | ) 217 | 218 | # Delete Secrets manager policy only if it was created (i.e. for Confluence, SharePoint or Salesforce data source) 219 | if sm_policy_flag: 220 | iam_client.detach_role_policy( 221 | RoleName=bedrock_execution_role_name, 222 | PolicyArn=sm_policy_arn 223 | ) 224 | iam_client.delete_policy(PolicyArn=sm_policy_arn) 225 | iam_client.delete_role(RoleName=bedrock_execution_role_name) 226 | iam_client.delete_policy(PolicyArn=s3_policy_arn) 227 | iam_client.delete_policy(PolicyArn=fm_policy_arn) 228 | iam_client.delete_policy(PolicyArn=oss_policy_arn) 229 | 230 | return 0 231 | 232 | 233 | def interactive_sleep(seconds: int): 234 | dots = '' 235 | for i in range(seconds): 236 | dots += '.' 237 | print(dots, end='\r') 238 | time.sleep(1) 239 | 240 | def create_bedrock_execution_role_multi_ds(bucket_names = None, secrets_arns = None): 241 | 242 | # 0. Create bedrock execution role 243 | 244 | assume_role_policy_document = { 245 | "Version": "2012-10-17", 246 | "Statement": [ 247 | { 248 | "Effect": "Allow", 249 | "Principal": { 250 | "Service": "bedrock.amazonaws.com" 251 | }, 252 | "Action": "sts:AssumeRole" 253 | } 254 | ] 255 | } 256 | 257 | # create bedrock execution role 258 | bedrock_kb_execution_role = iam_client.create_role( 259 | RoleName=bedrock_execution_role_name, 260 | AssumeRolePolicyDocument=json.dumps(assume_role_policy_document), 261 | Description='Amazon Bedrock Knowledge Base Execution Role for accessing OSS, secrets manager and S3', 262 | MaxSessionDuration=3600 263 | ) 264 | 265 | # fetch arn of the role created above 266 | bedrock_kb_execution_role_arn = bedrock_kb_execution_role['Role']['Arn'] 267 | 268 | # 1. Cretae and attach policy for foundation models 269 | foundation_model_policy_document = { 270 | "Version": "2012-10-17", 271 | "Statement": [ 272 | { 273 | "Effect": "Allow", 274 | "Action": [ 275 | "bedrock:InvokeModel", 276 | ], 277 | "Resource": [ 278 | f"arn:aws:bedrock:{region_name}::foundation-model/amazon.titan-embed-text-v1", 279 | f"arn:aws:bedrock:{region_name}::foundation-model/amazon.titan-embed-text-v2:0" 280 | ] 281 | } 282 | ] 283 | } 284 | 285 | fm_policy = iam_client.create_policy( 286 | PolicyName=fm_policy_name, 287 | PolicyDocument=json.dumps(foundation_model_policy_document), 288 | Description='Policy for accessing foundation model', 289 | ) 290 | 291 | # fetch arn of this policy 292 | fm_policy_arn = fm_policy["Policy"]["Arn"] 293 | 294 | # attach this policy to Amazon Bedrock execution role 295 | iam_client.attach_role_policy( 296 | RoleName=bedrock_kb_execution_role["Role"]["RoleName"], 297 | PolicyArn=fm_policy_arn 298 | ) 299 | 300 | # 2. Cretae and attach policy for s3 bucket 301 | if bucket_names: 302 | s3_policy_document = { 303 | "Version": "2012-10-17", 304 | "Statement": [ 305 | { 306 | "Effect": "Allow", 307 | "Action": [ 308 | "s3:GetObject", 309 | "s3:ListBucket" 310 | ], 311 | "Resource": [item for sublist in [[f'arn:aws:s3:::{bucket}', f'arn:aws:s3:::{bucket}/*'] for bucket in bucket_names] for item in sublist], 312 | "Condition": { 313 | "StringEquals": { 314 | "aws:ResourceAccount": f"{account_number}" 315 | } 316 | } 317 | } 318 | ] 319 | } 320 | # create policies based on the policy documents 321 | s3_policy = iam_client.create_policy( 322 | PolicyName=s3_policy_name, 323 | PolicyDocument=json.dumps(s3_policy_document), 324 | Description='Policy for reading documents from s3') 325 | 326 | # fetch arn of this policy 327 | s3_policy_arn = s3_policy["Policy"]["Arn"] 328 | 329 | # attach this policy to Amazon Bedrock execution role 330 | iam_client.attach_role_policy( 331 | RoleName=bedrock_kb_execution_role["Role"]["RoleName"], 332 | PolicyArn=s3_policy_arn 333 | ) 334 | 335 | # 3. Cretae and attach policy for secrets manager 336 | if secrets_arns: 337 | sm_policy_flag = True 338 | secrets_manager_policy_document = { 339 | "Version": "2012-10-17", 340 | "Statement": [ 341 | { 342 | "Effect": "Allow", 343 | "Action": [ 344 | "secretsmanager:GetSecretValue", 345 | "secretsmanager:PutSecretValue" 346 | ], 347 | "Resource": secrets_arns 348 | } 349 | ] 350 | } 351 | # create policies based on the policy documents 352 | 353 | secrets_manager_policy = iam_client.create_policy( 354 | PolicyName=sm_policy_name, 355 | PolicyDocument=json.dumps(secrets_manager_policy_document), 356 | Description='Policy for accessing secret manager', 357 | ) 358 | 359 | # fetch arn of this policy 360 | sm_policy_arn = secrets_manager_policy["Policy"]["Arn"] 361 | 362 | # attach policy to Amazon Bedrock execution role 363 | iam_client.attach_role_policy( 364 | RoleName=bedrock_kb_execution_role["Role"]["RoleName"], 365 | PolicyArn=sm_policy_arn 366 | ) 367 | 368 | return bedrock_kb_execution_role 369 | -------------------------------------------------------------------------------- /03_Model_customization/README.md: -------------------------------------------------------------------------------- 1 | # Lab 10 - Custom Models 2 | 3 | 4 |
5 | Warning: This module cannot be executed in Workshop Studio Accounts, and you will have to run this notebook in your own account. 6 |
7 | 8 | 9 | ## Overview 10 | Model customization is the process of providing training data to a model in order to improve its performance for specific use-cases. You can customize Amazon Bedrock foundation models in order to improve their performance and create a better customer experience. Amazon Bedrock currently provides the following customization methods. 11 | 12 | - Fine-tuning 13 | 14 | Provide labeled data in order to train a model to improve performance on specific tasks. By providing a training dataset of labeled examples, the model learns to associate what types of outputs should be generated for certain types of inputs. The model parameters are adjusted in the process and the model's performance is improved for the tasks represented by the training dataset. 15 | 16 | - Continued Pre-training 17 | 18 | Provide unlabeled data to pre-train a foundation model by familiarizing it with certain types of inputs. You can provide data from specific topics in order to expose a model to those areas. The Continued Pre-training process will tweak the model parameters to accommodate the input data and improve its domain knowledge. For example, you can train a model with private data, such as business documents, that are not publically available for training large language models. Additionally, you can continue to improve the model by retraining the model with more unlabeled data as it becomes available. 19 | 20 | ## Relevance 21 | Using your own data, you can privately and securely customize foundation models (FMs) in Amazon Bedrock to build applications that are specific to your domain, organization, and use case. Custom models enable you to create unique user experiences that reflect your company’s style, voice, and services. 22 | 23 | - With fine-tuning, you can increase model accuracy by providing your own task-specific labeled training dataset and further specialize your FMs. 24 | - With continued pre-training, you can train models using your own unlabeled data in a secure and managed environment with customer managed keys. Continued pre-training helps models become more domain-specific by accumulating more robust knowledge and adaptability—beyond their original training. 25 | 26 | This module walks you through how to customize models through fine-tuning and continued pre-training, how to provision the custom models with provisioned throughput, and how to compare and evaluate model performance. 27 | 28 | ## Target Audience 29 | 30 | This module can be executed by any developer familiar with Python, also by data scientists and other technical people who aspire to customize FMs in Bedrock. 31 | 32 | ## Setup 33 | - In this module, please run the the 01_setup.ipynb notebook first to make sure resources are properly set up for the following notebooks in this lab. 34 | - At the end of the module, please run the 03_cleanup.ipynb to make sure resources are removed to avoid unnecessary costs. 35 | 36 | 37 | ## Patterns 38 | 39 | In this workshop, you will be able to learn following patterns on customizing FMs in Bedrock: 40 | 41 | 42 | # Fine tuning - 43 | 44 | 1. [Fine-tune and Evaluate Amazon Nova in Bedrock ](./bedrock-models-fine-tuning/amazon-nova/01_Amazon_Nova_Finetuning_Walkthrough.ipynb): Demonstrates an end-to-end workflow for fine-tuning, provisioning and evaluating a Amazon Nova in Amazon Bedrock. 45 | 2. [Fine-tune and Evaluate Claude Haiku in Bedrock](./bedrock-models-fine-tuning/claude-haiku/02_fine-tune_Claude_Haiku.ipynb): Demonstrates an end-to-end workflow for fine-tuning, provisioning and evaluating a Claude Haiku in Amazon Bedrock. 46 | 3. [Fine-tune and Evaluate Meta Llama 3 in Bedrock](./bedrock-models-fine-tuning/meta-llama/Llama-3.2%20Multi-modal%20cusotmization/02_fine-tune_llama3.2.ipynb): Demonstrates an end-to-end workflow for fine-tuning, provisioning and evaluating a Meta Llama 3.2 multimodal customization in Amazon Bedrock. 47 | 48 | # Continued Pretraining - 49 | 50 | 1. [Continued Pretraining with Amazon Titan ](./continued%20Pre-training/02_continued_pretraining_titan_text.ipynb): Demonstrates an end-to-end workflow for continued pretraining Amazon Titan model in Amazon Bedrock. -------------------------------------------------------------------------------- /03_Model_customization/bedrock-models-fine-tuning/README.md: -------------------------------------------------------------------------------- 1 | # Fine Tuning 2 | 3 | This folder contains examples related to Fine-tuning Bedrock Models 4 | 5 | ## Contents 6 | 7 | This folder includes examples to setup, fine-tune and evaluate three model families on Amazon Bedrock. 8 | 9 | # 1. Amazon Nova 10 | Model - Amazon Nova Lite and Amazon Nova Micro model using Amazon Bedrock 11 | # 2. Meta Llama 12 | Model - Meta Llama 3.2 with vision capabilities using Amazon Bedrock 13 | # 3. Claude Haiku 14 | Model - Anthropic Claude-3 Haiku model using Amazon Bedrock 15 | 16 | 17 | ## Contributing 18 | 19 | We welcome community contributions! Please ensure your sample aligns with AWS [best practices](https://aws.amazon.com/architecture/well-architected/), and please update the **Contents** section of this README file with a link to your sample, along with a description. 20 | -------------------------------------------------------------------------------- /03_Model_customization/bedrock-models-fine-tuning/amazon-nova/01_fine-tune_Amazon_Nova.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0cadfd20", 6 | "metadata": {}, 7 | "source": [ 8 | "// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 9 | "// SPDX-License-Identifier: MIT-0" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "id": "d61daf89-0485-47bf-aa96-280577bbadaf", 15 | "metadata": {}, 16 | "source": [ 17 | "# Fine-Tune Amazon Nova model provided by Amazon Bedrock: End-to-End\n", 18 | "\n", 19 | "This notebook demonstrates the end-to-end process of fine-tuning Amazon Nova Lite and Amazon Nova Micro model using Amazon Bedrock, including selecting the base model, configuring hyperparameters, creating and monitoring the fine-tuning job, deploying the fine-tuned model with provisioned throughput and evaluating the performance of the fine-tuned model. \n", 20 | "\n", 21 | "Note: The following steps can also be done through the Amazon Bedrock Console" 22 | ] 23 | }, 24 | { 25 | "attachments": {}, 26 | "cell_type": "markdown", 27 | "id": "7cc3adc2", 28 | "metadata": {}, 29 | "source": [ 30 | "# Prerequisites\n", 31 | "\n", 32 | "- Make sure you have prepared a fine-tuning dataset following the format required [here]( \n", 33 | "https://docs.aws.amazon.com/nova/latest/userguide/customize-fine-tune-prepare.html)\n", 34 | "- Make sure your AWS account has appropriate permissions (e.g. access to Amazon Bedrock (us-east-1))" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": null, 40 | "id": "0147f270-3cf7-4a94-b806-9fc3aabda552", 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "!pip install -qU -r requirements.txt" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "id": "ef96a213-b08e-41ec-a987-c1fdd4e3aa0b", 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "# restart kernel for packages to take effect\n", 55 | "from IPython.core.display import HTML\n", 56 | "HTML(\"\")" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "id": "dae87342-ea92-4b49-a705-184027a76356", 62 | "metadata": {}, 63 | "source": [ 64 | "# Setup" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": 3, 70 | "id": "2e1cbb30-eedc-4079-a8b3-5eb21dd88e53", 71 | "metadata": { 72 | "tags": [] 73 | }, 74 | "outputs": [], 75 | "source": [ 76 | "import boto3 \n", 77 | "from botocore.config import Config\n", 78 | "import sys\n", 79 | "import pandas as pd\n", 80 | "import matplotlib.pyplot as plt\n", 81 | "import json\n", 82 | "import time \n", 83 | "import concurrent.futures\n", 84 | "import shortuuid\n", 85 | "import tqdm\n", 86 | "import os" 87 | ] 88 | }, 89 | { 90 | "cell_type": "code", 91 | "execution_count": 4, 92 | "id": "742d8e7f-bb20-4283-9b81-fd67fb15c88d", 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "my_config = Config(\n", 97 | " region_name = 'us-east-1', \n", 98 | " signature_version = 'v4',\n", 99 | " retries = {\n", 100 | " 'max_attempts': 5,\n", 101 | " 'mode': 'standard'\n", 102 | " })\n", 103 | "\n", 104 | "bedrock = boto3.client(service_name=\"bedrock\", config=my_config)" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": null, 110 | "id": "eedbd33d", 111 | "metadata": {}, 112 | "outputs": [], 113 | "source": [ 114 | "## Specify input S3 bucket\n", 115 | "\n", 116 | "input_s3_uri = \"s3://ft-aws-domain/ft_bedrock_data/ft_olympus_jsonl/aws_train_olympus.jsonl\"\n", 117 | "output_s3_uri = \"s3://ft-aws-domain/model_output/ft_nova_lite_aws_v1/\"" 118 | ] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "id": "41d127c7-8bee-4cca-86af-cf624a088ad0", 123 | "metadata": {}, 124 | "source": [ 125 | "# Select the base model to fine-tune" 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "id": "1556c1dd", 131 | "metadata": {}, 132 | "source": [ 133 | "You need to provide the `base_model_id` for the model you want to fine-tune. You can find a list of the foundational model ids by invoking the `list_foundation_models` API:\n", 134 | "\n", 135 | "``` \\n\n", 136 | "for model in bedrock.list_foundation_models(\n", 137 | " byCustomizationType=\"FINE_TUNING\")[\"modelSummaries\"]:\n", 138 | " for key, value in model.items():\n", 139 | " print(key, \":\", value)\n", 140 | " print(\"-----\\n\")\n", 141 | "```" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": 24, 147 | "id": "33b0440f-838d-412e-bcf0-bf3ca9050aa8", 148 | "metadata": { 149 | "tags": [] 150 | }, 151 | "outputs": [], 152 | "source": [ 153 | "nova_micro_identifier = \"arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-micro-v1:0:128k\"\n", 154 | "nova_lite_identifier = \"arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-lite-v1:0:300k\"" 155 | ] 156 | }, 157 | { 158 | "cell_type": "markdown", 159 | "id": "782da1f9", 160 | "metadata": {}, 161 | "source": [ 162 | "Next, provide the `customization_job_name`, `custom_model_name` and `customization_role` which will be used to create the fine-tuning job." 163 | ] 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": null, 168 | "id": "4ae02645", 169 | "metadata": {}, 170 | "outputs": [], 171 | "source": [ 172 | "# Nova model customization currently only available in US EAST 1\n", 173 | "\n", 174 | "role_name = # Replace with your role name\n", 175 | "role_arn = # Replace with your role ARN\n", 176 | "\n", 177 | "job_name = \"aws-ft-nova-lite-v1\"\n", 178 | "model_name = job_name " 179 | ] 180 | }, 181 | { 182 | "cell_type": "markdown", 183 | "id": "f57903f8-0c1c-4039-b398-45182d937174", 184 | "metadata": {}, 185 | "source": [ 186 | "# Create fine-tuning job" 187 | ] 188 | }, 189 | { 190 | "cell_type": "markdown", 191 | "id": "13ca815f-255f-4920-87b9-acb36b27094d", 192 | "metadata": {}, 193 | "source": [ 194 | "
\n", 195 | " Note: Fine-tuning job will take around 2-4 hrs to complete.
" 196 | ] 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "id": "8e57db8e-5356-4ccd-8d53-35941d4db999", 201 | "metadata": {}, 202 | "source": [ 203 | "| ***Parameter Name*** | ***Parameter Description*** | ***Type*** | ***Min*** | ***Max*** | **Default** |\n", 204 | "| ------- | ------------- | ------ | --------- | ----------- | ----------- |\n", 205 | "| Epochs | The maximum number of iterations through the entire training dataset | integer | 1 | 5 | 2 |\n", 206 | "| Learning rate | The rate at which model parameters are updated after each batch of training data | float | 1.00E-06 | 1.00E-04 | 1.00E-05 |\n", 207 | "| Learning rate warmup steps | Number of iterations over which learning rate is gradually increased to the initial rate specified | integer | 0 | 20 | 10 |\n", 208 | "| Batch size | The number of samples processed before updating model parameters | integer | NA | NA | Fixed at 1|" 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": 25, 214 | "id": "0ee8a157-fb04-4e9d-beca-bdabfbed39e7", 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [ 218 | "# Select the customization type from \"FINE_TUNING\" or \"CONTINUED_PRE_TRAINING\". \n", 219 | "customization_type = \"FINE_TUNING\"\n", 220 | "\n", 221 | "\n", 222 | "# Define the hyperparameters for fine-tuning Amazon Nova model\n", 223 | "hyper_parameters = {\n", 224 | " \"epochCount\": \"1\",\n", 225 | " \"learningRate\": '0.000001', \n", 226 | " \"batchSize\": \"1\",\n", 227 | " }\n", 228 | "\n", 229 | "\n", 230 | "response_ft = bedrock.create_model_customization_job(\n", 231 | " customizationType=customization_type,\n", 232 | " jobName = job_name,\n", 233 | " customModelName = model_name,\n", 234 | " roleArn = role_arn,\n", 235 | " baseModelIdentifier = nova_lite_identifier,\n", 236 | " hyperParameters=hyper_parameters,\n", 237 | " trainingDataConfig={\"s3Uri\": input_s3_uri},\n", 238 | " outputDataConfig={\"s3Uri\": output_s3_uri},\n", 239 | ")" 240 | ] 241 | }, 242 | { 243 | "cell_type": "markdown", 244 | "id": "27f1137d-3ba8-4d42-af84-6d1e489af009", 245 | "metadata": {}, 246 | "source": [ 247 | "# Check fine-tuning job status" 248 | ] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": 26, 253 | "id": "99ccd7de-ae97-45d2-92b1-5f61e4157204", 254 | "metadata": {}, 255 | "outputs": [ 256 | { 257 | "name": "stdout", 258 | "output_type": "stream", 259 | "text": [ 260 | "Job status: InProgress\n" 261 | ] 262 | } 263 | ], 264 | "source": [ 265 | "jobArn = response_ft.get('jobArn')\n", 266 | "status = bedrock.get_model_customization_job(jobIdentifier=jobArn)[\"status\"]\n", 267 | "print(f'Job status: {status}')" 268 | ] 269 | }, 270 | { 271 | "cell_type": "markdown", 272 | "id": "54a8ac36-7e3d-4e51-94ef-dd41cd4fe21d", 273 | "metadata": {}, 274 | "source": [ 275 | "# Setup provisioned throughput" 276 | ] 277 | }, 278 | { 279 | "cell_type": "markdown", 280 | "id": "c11f6925", 281 | "metadata": {}, 282 | "source": [ 283 | "Once the job status changes to `complete`, we need to create provisioned throughput which is needed for running inference on the fine-tuned Amazon Nova model. For more information on provisioned throughput, please refer to [this documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html)" 284 | ] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "execution_count": null, 289 | "id": "71f64472-00ca-4f4f-a4d4-065e2b61684c", 290 | "metadata": {}, 291 | "outputs": [], 292 | "source": [ 293 | "provisioned_model_name = 'finetuned_nova_lite'\n", 294 | "custom_model_id = 'aws-ft-nova-lite-v1'\n", 295 | "\n", 296 | "provisioned_model_id = bedrock.create_provisioned_model_throughput(\n", 297 | " modelUnits=1,\n", 298 | " provisionedModelName=provisioned_model_name,\n", 299 | " modelId=custom_model_id\n", 300 | " )\n", 301 | "\n", 302 | "print(provisioned_model_id['provisionedModelArn'])" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": null, 308 | "id": "de9f56bd", 309 | "metadata": {}, 310 | "outputs": [], 311 | "source": [ 312 | "status_provisioning = bedrock.get_provisioned_model_throughput(provisionedModelId = provisioned_model_id)['status']\n", 313 | "\n", 314 | "import time\n", 315 | "while status_provisioning == 'Creating':\n", 316 | " time.sleep(60)\n", 317 | " status_provisioning = bedrock.get_provisioned_model_throughput(provisionedModelId=provisioned_model_id)['status']\n", 318 | " print(status_provisioning)\n", 319 | " time.sleep(60)" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "id": "d6a8606f-049a-4c7b-939c-c5c44bfda210", 325 | "metadata": {}, 326 | "source": [ 327 | "# Delete provisioned throughput" 328 | ] 329 | }, 330 | { 331 | "cell_type": "markdown", 332 | "id": "b9939990", 333 | "metadata": {}, 334 | "source": [ 335 | "Warning: Please make sure to delete providsioned throughput as there will cost incurred if its left in running state, even if you are not using it." 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": null, 341 | "id": "4341f7e8-e85b-4c88-9f77-11b2afc463de", 342 | "metadata": {}, 343 | "outputs": [], 344 | "source": [ 345 | "bedrock.delete_provisioned_model_throughput(provisionedModelId=provisioned_model_id)" 346 | ] 347 | } 348 | ], 349 | "metadata": { 350 | "kernelspec": { 351 | "display_name": "conda_pytorch_p310", 352 | "language": "python", 353 | "name": "conda_pytorch_p310" 354 | }, 355 | "language_info": { 356 | "codemirror_mode": { 357 | "name": "ipython", 358 | "version": 3 359 | }, 360 | "file_extension": ".py", 361 | "mimetype": "text/x-python", 362 | "name": "python", 363 | "nbconvert_exporter": "python", 364 | "pygments_lexer": "ipython3", 365 | "version": "3.10.14" 366 | } 367 | }, 368 | "nbformat": 4, 369 | "nbformat_minor": 5 370 | } 371 | -------------------------------------------------------------------------------- /03_Model_customization/bedrock-models-fine-tuning/amazon-nova/02_Inference_Amazon_Nova.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "b18c5412", 6 | "metadata": {}, 7 | "source": [ 8 | "// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.\n", 9 | "// SPDX-License-Identifier: MIT-0" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "id": "d61daf89-0485-47bf-aa96-280577bbadaf", 15 | "metadata": {}, 16 | "source": [ 17 | "# Inference with Customized Amazon Nova Models \n", 18 | "\n", 19 | "This notebook walk-through how to conduct inference on fine-tuned Amazon Nova models. We first demonstrate a single example followed by example scripts for running batch inference." 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "id": "3cd5400c", 25 | "metadata": {}, 26 | "source": [ 27 | "# Prerequisites\n", 28 | "\n", 29 | "- Make sure you have executed 01_Amazon_Nova_Finetuning_Walkthrough.ipynb notebook.\n", 30 | "- Make sure you are using the same kernel and instance as 01_Amazon_Nova_Finetuning_Walkthrough.ipynb notebook." 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "id": "0147f270-3cf7-4a94-b806-9fc3aabda552", 37 | "metadata": {}, 38 | "outputs": [], 39 | "source": [ 40 | "!pip install -qU -r requirements.txt" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 13, 46 | "id": "ef96a213-b08e-41ec-a987-c1fdd4e3aa0b", 47 | "metadata": {}, 48 | "outputs": [ 49 | { 50 | "data": { 51 | "text/html": [ 52 | "" 53 | ], 54 | "text/plain": [ 55 | "" 56 | ] 57 | }, 58 | "execution_count": 13, 59 | "metadata": {}, 60 | "output_type": "execute_result" 61 | } 62 | ], 63 | "source": [ 64 | "# restart kernel for packages to take effect\n", 65 | "from IPython.core.display import HTML\n", 66 | "HTML(\"\")" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "id": "dae87342-ea92-4b49-a705-184027a76356", 72 | "metadata": {}, 73 | "source": [ 74 | "# Setup" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 45, 80 | "id": "2e1cbb30-eedc-4079-a8b3-5eb21dd88e53", 81 | "metadata": { 82 | "tags": [] 83 | }, 84 | "outputs": [], 85 | "source": [ 86 | "import boto3 \n", 87 | "from botocore.config import Config\n", 88 | "import sys\n", 89 | "import pandas as pd\n", 90 | "import matplotlib.pyplot as plt\n", 91 | "import json\n", 92 | "import time \n", 93 | "import concurrent.futures\n", 94 | "import shortuuid\n", 95 | "import tqdm\n", 96 | "import os" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": 46, 102 | "id": "742d8e7f-bb20-4283-9b81-fd67fb15c88d", 103 | "metadata": {}, 104 | "outputs": [], 105 | "source": [ 106 | "my_config = Config(\n", 107 | " region_name = 'us-east-1', \n", 108 | " signature_version = 'v4',\n", 109 | " retries = {\n", 110 | " 'max_attempts': 5,\n", 111 | " 'mode': 'standard'\n", 112 | " })\n", 113 | "\n", 114 | "bedrock = boto3.client(service_name=\"bedrock\", config=my_config)" 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "id": "4d10ca06-93a4-4552-aec4-6684ff9d2464", 120 | "metadata": {}, 121 | "source": [ 122 | "# Construct model input " 123 | ] 124 | }, 125 | { 126 | "cell_type": "markdown", 127 | "id": "a8bb3880", 128 | "metadata": {}, 129 | "source": [ 130 | "Before invoking the customized models, we need to construct model input following the format needed by Amazon Nova models." 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": 42, 136 | "id": "abc59170-bc16-466b-a78e-92e13972108e", 137 | "metadata": {}, 138 | "outputs": [], 139 | "source": [ 140 | "# API setting constants\n", 141 | "API_MAX_RETRY = 16\n", 142 | "API_RETRY_SLEEP = 10\n", 143 | "API_ERROR_OUTPUT = \"$ERROR$\"\n", 144 | "\n", 145 | "\n", 146 | "def create_nova_messages(prompt):\n", 147 | " \"\"\"\n", 148 | " Create messages array for Amazon Nova models from conversation\n", 149 | "\n", 150 | " Args:\n", 151 | " conv (object): Conversation object containing messages\n", 152 | "\n", 153 | " Returns:\n", 154 | " list: List of formatted messages for Amazon Nova model\n", 155 | " \"\"\"\n", 156 | " messages = []\n", 157 | " \n", 158 | " messages.append({\n", 159 | " \"role\": \"user\",\n", 160 | " \"content\": [{\"text\": prompt}]\n", 161 | " })\n", 162 | "\n", 163 | " return messages\n", 164 | "\n", 165 | "def chat_completion_aws_bedrock_nova(model, conv, temperature, max_tokens, aws_region=\"us-east-1\"):\n", 166 | " \"\"\"\n", 167 | " Call AWS Bedrock API for chat completion using Amazon Nova models\n", 168 | "\n", 169 | " Args:\n", 170 | " model (str): Model ID\n", 171 | " conv (object): Conversation object containing messages\n", 172 | " temperature (float): Temperature parameter for response generation\n", 173 | " max_tokens (int): Maximum tokens in response\n", 174 | " api_dict (dict, optional): API configuration dictionary\n", 175 | " aws_region (str, optional): AWS region, defaults to \"us-west-2\"\n", 176 | "\n", 177 | " Returns:\n", 178 | " str: Generated response text or error message\n", 179 | " \"\"\"\n", 180 | "\n", 181 | " # Configure AWS client \n", 182 | " bedrock_rt_client = boto3.client(\n", 183 | " service_name='bedrock-runtime',\n", 184 | " region_name=aws_region,\n", 185 | " )\n", 186 | "\n", 187 | " \n", 188 | " # Retry logic for API calls\n", 189 | " for _ in range(API_MAX_RETRY):\n", 190 | " try:\n", 191 | " # Create messages from conversation\n", 192 | " messages = create_nova_messages(conv)\n", 193 | " inferenceConfig = {\n", 194 | " \"max_new_tokens\": max_tokens,\n", 195 | " \"temperature\": temperature, \n", 196 | " }\n", 197 | "\n", 198 | " # Prepare request body\n", 199 | " model_kwargs = {\"messages\": messages,\n", 200 | " \"inferenceConfig\": inferenceConfig}\n", 201 | " body = json.dumps(model_kwargs)\n", 202 | "\n", 203 | " # Call Bedrock API\n", 204 | " response = bedrock_rt_client.invoke_model(\n", 205 | " body=body,\n", 206 | " modelId=model,\n", 207 | " accept='application/json',\n", 208 | " contentType='application/json'\n", 209 | " )\n", 210 | "\n", 211 | " # Parse response\n", 212 | " response_body = json.loads(response.get('body').read())\n", 213 | " \n", 214 | " output = response_body['output']['message']['content'][0]['text']\n", 215 | " break\n", 216 | "\n", 217 | " except Exception as e:\n", 218 | " print(type(e), e)\n", 219 | " ## Uncomment time.sleep if encounter Bedrock invoke throttling error\n", 220 | " # time.sleep(API_RETRY_SLEEP)\n", 221 | "\n", 222 | " return output" 223 | ] 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "id": "7342ceae-3f3d-4c4c-b2cb-967a2df68483", 228 | "metadata": {}, 229 | "source": [ 230 | "# Inference on customized Amazon Nova model (individual example)" 231 | ] 232 | }, 233 | { 234 | "cell_type": "code", 235 | "execution_count": 51, 236 | "id": "485a3c00", 237 | "metadata": {}, 238 | "outputs": [ 239 | { 240 | "name": "stdout", 241 | "output_type": "stream", 242 | "text": [ 243 | "When anonymous operational metrics are enabled for an Amazon Elastic File System (EFS) file system, AWS collects and sends specific details to Amazon CloudWatch. These metrics are designed to help you monitor the performance and usage of your EFS file systems without requiring you to provide personally identifiable information. Here are the specific details that are collected and sent:\n", 244 | "\n", 245 | "1. **File System-Level Metrics**:\n", 246 | " - **FileSystemSize**: The total size of the file system in bytes.\n", 247 | " - **FreeStorageCapacity**: The amount of available storage capacity in bytes.\n", 248 | " - **BurstingCredits**: The number of bursting credits available for the file system.\n", 249 | " - **BurstBalance**: The current balance of bursting credits.\n", 250 | " - **ThroughputMode**: The throughput mode of the file system (e.g., Bursting, Provisioned).\n", 251 | "\n", 252 | "2. **Network Metrics**:\n", 253 | " - **NetworkThroughput**: The amount of network throughput in bytes per second.\n", 254 | " - **NetworkLatency**: The average latency of network requests in milliseconds.\n", 255 | "\n", 256 | "3. **I/O Metrics**:\n", 257 | " - **ReadOperations**: The number of read operations.\n", 258 | " - **WriteOperations**: The number of write operations.\n", 259 | " - **TotalIOPS**: The total input/output operations per second.\n", 260 | " - **ReadIOPS**: The number of read IOPS.\n", 261 | " - **WriteIOPS**: The number of write IOPS.\n", 262 | "\n", 263 | "4. **Mount Target Metrics**:\n", 264 | " - **NetworkThroughputPerMountTarget**: The network throughput per mount target in bytes per second.\n", 265 | " - **NetworkLatencyPerMountTarget**: The average network latency per mount target in milliseconds.\n", 266 | "\n", 267 | "5. **Additional Metrics**:\n", 268 | " - **MountTargetCount**: The number of mount targets associated with the file system.\n", 269 | " - **BackupSize**: The size of the backup in bytes (if backups are enabled).\n", 270 | "\n", 271 | "These metrics are sent to CloudWatch in 1-minute intervals, allowing you to monitor and analyze the performance and usage of your EFS file systems over time. The data is anonymized and aggregated to ensure that no personally identifiable information is shared.\n", 272 | "\n", 273 | "To enable these metrics, you can configure your EFS file system to send anonymous operational data to AWS through the AWS Management Console, AWS CLI, or AWS SDKs. Once enabled, AWS will start collecting and sending these metrics to CloudWatch automatically.\n" 274 | ] 275 | } 276 | ], 277 | "source": [ 278 | "# [Important!] Update `base_model_id` to `provisioned_model_id` based on the previous jupyter notebook\n", 279 | "base_model_id = 'amazon.nova-lite-v1:0'\n", 280 | "temperature = 0.2\n", 281 | "max_tokens = 1024\n", 282 | "\n", 283 | "ques = \"What specific details are collected and sent to AWS when anonymous operational metrics are enabled for an Amazon EFS file system?\"\n", 284 | "\n", 285 | "print(chat_completion_aws_bedrock_nova(base_model_id, ques, temperature+0.01, max_tokens, aws_region=\"us-east-1\")) " 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "id": "731bbeea", 291 | "metadata": {}, 292 | "source": [ 293 | "# Batch inference with customized Amazon Nova model " 294 | ] 295 | }, 296 | { 297 | "cell_type": "markdown", 298 | "id": "21c027cf", 299 | "metadata": {}, 300 | "source": [ 301 | "In this section, we provide code snippets for efficiently running batch inference using the same `chat_completion_aws_bedrock_nova` function as above." 302 | ] 303 | }, 304 | { 305 | "cell_type": "code", 306 | "execution_count": 52, 307 | "id": "f0c444dd", 308 | "metadata": { 309 | "tags": [] 310 | }, 311 | "outputs": [ 312 | { 313 | "name": "stdout", 314 | "output_type": "stream", 315 | "text": [ 316 | "[' \"What specific details are collected and sent to AWS when anonymous operational metrics are enabled for an Amazon EFS file system?', \"What's required for a successful AWS CloudFormation launch?\"]\n" 317 | ] 318 | } 319 | ], 320 | "source": [ 321 | "# Load test cases \n", 322 | "question_file = f\"dataset/test_set/question_short.jsonl\"\n", 323 | "\n", 324 | "questions = []\n", 325 | "with open(question_file, \"r\", encoding=\"utf-8\") as ques_file:\n", 326 | " for line in ques_file:\n", 327 | " if line:\n", 328 | " questions.append(json.loads(line))\n", 329 | "\n", 330 | "print(questions[0][\"turns\"])" 331 | ] 332 | }, 333 | { 334 | "cell_type": "code", 335 | "execution_count": 53, 336 | "id": "65ff0707", 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [ 340 | "# Helper function that helps organize answers from customized Amazon Nova model\n", 341 | "\n", 342 | "def get_answer(\n", 343 | " question: dict, model_id: str, num_choices: int, max_tokens: int, temperature: float, answer_file: str\n", 344 | "):\n", 345 | "\n", 346 | " choices = []\n", 347 | "\n", 348 | " for i in range(num_choices):\n", 349 | " conv = \"\"\n", 350 | " turns = []\n", 351 | " \n", 352 | " for j in range(len(question[\"turns\"])):\n", 353 | " conv += question[\"turns\"][j]\n", 354 | " output = chat_completion_aws_bedrock_nova(model_id, conv, temperature+0.01, max_tokens, aws_region=\"us-east-1\") \n", 355 | " turns.append(output)\n", 356 | "\n", 357 | " choices.append({\"index\": i, \"turns\": turns})\n", 358 | "\n", 359 | " # Dump answers\n", 360 | " ans = {\n", 361 | " \"question_id\": question[\"question_id\"],\n", 362 | " \"answer_id\": shortuuid.uuid(),\n", 363 | " \"model_id\": model,\n", 364 | " 'use_rag': False,\n", 365 | " \"choices\": choices,\n", 366 | " \"tstamp\": time.time(),\n", 367 | " }\n", 368 | "\n", 369 | " os.makedirs(os.path.dirname(answer_file), exist_ok=True)\n", 370 | " with open(answer_file, \"a\", encoding=\"utf-8\") as f:\n", 371 | " f.write(json.dumps(ans) + \"\\n\")" 372 | ] 373 | }, 374 | { 375 | "cell_type": "code", 376 | "execution_count": 54, 377 | "id": "d97c7f3a-8640-4aa1-b11a-73b8053101e9", 378 | "metadata": { 379 | "tags": [] 380 | }, 381 | "outputs": [ 382 | { 383 | "name": "stdout", 384 | "output_type": "stream", 385 | "text": [ 386 | "Output to dataset/model_answer/amazon.nova-lite-v1:0_V2.jsonl\n" 387 | ] 388 | }, 389 | { 390 | "name": "stderr", 391 | "output_type": "stream", 392 | "text": [ 393 | "100%|██████████| 10/10 [01:19<00:00, 8.00s/it]\n" 394 | ] 395 | } 396 | ], 397 | "source": [ 398 | "# Run batch inference and save model output\n", 399 | "\n", 400 | "## [Important!] Update `base_model_id` to `provisioned_model_id` based on the previous jupyter notebook\n", 401 | "model_id = 'amazon.nova-lite-v1:0'\n", 402 | "num_choices = 1 \n", 403 | "max_tokens = 1024\n", 404 | "temperature = 0.2\n", 405 | " \n", 406 | "answer_file = f\"dataset/model_answer/{model_id}_V2.jsonl\"\n", 407 | "print(f\"Output to {answer_file}\")\n", 408 | "\n", 409 | "with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:\n", 410 | " futures = []\n", 411 | " for question in questions:\n", 412 | " future = executor.submit(\n", 413 | " get_answer,\n", 414 | " question,\n", 415 | " model_id,\n", 416 | " num_choices,\n", 417 | " max_tokens,\n", 418 | " temperature,\n", 419 | " answer_file,\n", 420 | " )\n", 421 | " futures.append(future)\n", 422 | "\n", 423 | " for future in tqdm.tqdm(\n", 424 | " concurrent.futures.as_completed(futures), total=len(futures)\n", 425 | " ):\n", 426 | " future.result()" 427 | ] 428 | }, 429 | { 430 | "cell_type": "markdown", 431 | "id": "ed39ae22-f78c-4cf6-8746-cc12aea739e9", 432 | "metadata": {}, 433 | "source": [ 434 | "# [Optional] Plot training loss" 435 | ] 436 | }, 437 | { 438 | "cell_type": "markdown", 439 | "id": "20cb7c64", 440 | "metadata": {}, 441 | "source": [ 442 | "Optionally, you can also plot training loss using the `step_wise_training_metrics.csv` file generated from the finetuning job. This csv file and other model artifacts can be found under Amazon Bedrock -> Custom model -> Custom model name -> Output data (S3 location) " 443 | ] 444 | }, 445 | { 446 | "cell_type": "code", 447 | "execution_count": null, 448 | "id": "54362a24-9222-412f-b89e-60913fbb52dd", 449 | "metadata": { 450 | "tags": [] 451 | }, 452 | "outputs": [], 453 | "source": [ 454 | "def plot_training_loss(input_file, output_file):\n", 455 | " ''' This function plots training loss using the default model output file 'step_wise_training_metrics.csv' generated from the finetuning job'''\n", 456 | " \n", 457 | " # Read the CSV file\n", 458 | " df = pd.read_csv(input_file)\n", 459 | " \n", 460 | " # Create the plot\n", 461 | " plt.figure(figsize=(10, 6))\n", 462 | " plt.plot(df['step_number'], df['training_loss'], 'b-', linewidth=2)\n", 463 | " \n", 464 | " # Customize the plot\n", 465 | " plt.title('Training Loss vs Step Number', fontsize=14)\n", 466 | " plt.xlabel('Step Number', fontsize=12)\n", 467 | " plt.ylabel('Training Loss', fontsize=12)\n", 468 | " plt.grid(True, linestyle='--', alpha=0.7)\n", 469 | " \n", 470 | " # Add some padding to the axes\n", 471 | " plt.margins(x=0.02)\n", 472 | " \n", 473 | " # Save the plot\n", 474 | " plt.savefig(output_file, dpi=300, bbox_inches='tight')\n", 475 | " plt.close()\n", 476 | " \n", 477 | " print(f\"Plot saved as {output_file}\")\n", 478 | "\n", 479 | "\n", 480 | "# Example usage\n", 481 | "\n", 482 | "plot_training_loss(input_file = 'model_training_loss/aws-ft-nova-lite/step_wise_training_metrics_epoch5_lr_1e-06.csv', \n", 483 | " output_file = 'model_training_loss/aws-ft-nova-lite/training_loss_epoch5_lr_1e-06.png')\n", 484 | "\n" 485 | ] 486 | }, 487 | { 488 | "attachments": {}, 489 | "cell_type": "markdown", 490 | "id": "f1aa38b9", 491 | "metadata": {}, 492 | "source": [ 493 | "# Conclusion\n", 494 | "\n", 495 | "In this and last notebook, we provided a detailed walkthrough on how to fine-tune, host, and conduct inference with customized Amazon Nova through the Amazon Bedrock API. Please refer to the [guidelines](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-guidelines.html) for more tips on fine-tuning Amazon Nova models to meet your need." 496 | ] 497 | }, 498 | { 499 | "cell_type": "markdown", 500 | "id": "d6a8606f-049a-4c7b-939c-c5c44bfda210", 501 | "metadata": {}, 502 | "source": [ 503 | "# Delete provisioned throughput" 504 | ] 505 | }, 506 | { 507 | "cell_type": "markdown", 508 | "id": "2d98273e", 509 | "metadata": {}, 510 | "source": [ 511 | "Warning: Please make sure to delete providsioned throughput as there will cost incurred if its left in running state, even if you are not using it." 512 | ] 513 | }, 514 | { 515 | "cell_type": "code", 516 | "execution_count": null, 517 | "id": "4341f7e8-e85b-4c88-9f77-11b2afc463de", 518 | "metadata": {}, 519 | "outputs": [], 520 | "source": [ 521 | "bedrock.delete_provisioned_model_throughput(provisionedModelId=provisioned_model_id)" 522 | ] 523 | } 524 | ], 525 | "metadata": { 526 | "kernelspec": { 527 | "display_name": "conda_pytorch_p310", 528 | "language": "python", 529 | "name": "conda_pytorch_p310" 530 | }, 531 | "language_info": { 532 | "codemirror_mode": { 533 | "name": "ipython", 534 | "version": 3 535 | }, 536 | "file_extension": ".py", 537 | "mimetype": "text/x-python", 538 | "name": "python", 539 | "nbconvert_exporter": "python", 540 | "pygments_lexer": "ipython3", 541 | "version": "3.10.14" 542 | } 543 | }, 544 | "nbformat": 4, 545 | "nbformat_minor": 5 546 | } 547 | -------------------------------------------------------------------------------- /03_Model_customization/bedrock-models-fine-tuning/amazon-nova/README.md: -------------------------------------------------------------------------------- 1 | # amazon-nova-domain-specific-finetuning 2 | 3 | This repo provides example scripts for fine-tuning Amazon Nova models, hosting fine-tuned models through provisioned throughput and performing inference on MTbench style questions. 4 | 5 | ## Get-started 6 | 7 | ### Notebook walkthrough 8 | 9 | `01_fine-tune_Amazon_Nova.ipynb`: This notebook demonstrates the end-to-end process of fine-tuning Amazon Nova models using Amazon Bedrock, including selecting the base model, configuring hyperparameters, creating and monitoring the fine-tuning job, deploying the fine-tuned model with provisioned throughput and evaluating the performance of the fine-tuned model. 10 | 11 | `02_Inference_Amazon_Nova.ipynb`: This notebook walk-through how to conduct inference on fine-tuned Amazon Nova models. We first demonstrate a single example followed by example scripts for running batch inference. 12 | 13 | 14 | ## Installation 15 | Please install dependencies using the `requirements.txt` file 16 | `pip install -r requirements.txt` 17 | 18 | 19 | ## Support 20 | Reach out to florawan@amazon.com 21 | 22 | 23 | ## License 24 | This library is licensed under the MIT-0 License. See the [LICENSE](https://ssh.gitlab.aws.dev/genaiic-reusable-assets/demo-artifacts/olympus-domain-specific-finetuning/-/blob/main/LICENSE.txt) file. 25 | 26 | 27 | ## Disclaimer 28 | This repository is not production-ready and requires additional considerations on topics such as for security, reliability, and scalability before real-world deployment. -------------------------------------------------------------------------------- /03_Model_customization/bedrock-models-fine-tuning/amazon-nova/finetune-diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/03_Model_customization/bedrock-models-fine-tuning/amazon-nova/finetune-diagram.png -------------------------------------------------------------------------------- /03_Model_customization/bedrock-models-fine-tuning/amazon-nova/requirements.txt: -------------------------------------------------------------------------------- 1 | boto3 2 | transformers 3 | shortuuid 4 | matplotlib -------------------------------------------------------------------------------- /03_Model_customization/bedrock-models-fine-tuning/meta-llama/Llama-3.2 Multi-modal cusotmization/README.md: -------------------------------------------------------------------------------- 1 | # Fine-Tuning Llama 3.2 with Vision Capabilities using Amazon Bedrock 2 | 3 | This repository demonstrates how to fine-tune Meta's Llama 3.2 model with vision capabilities using Amazon Bedrock. Learn how to prepare multi-modal data, train a custom model, and run inference with your fine-tuned model. 4 | 5 | ## Contents 6 | 7 | - [01_setup_llama-3.2.ipynb](./01_setup_llama-3.2.ipynb) - Prepare data for fine-tuning Llama 3.2 with vision capabilities. This notebook covers downloading and processing a subset of the llava-instruct dataset with COCO images, formatting the data according to the Bedrock conversation schema, and preparing training, validation, and test datasets. 8 | 9 | - [/02_fine-tune_llama3.2.ipynb](./02_fine-tune_llama3.2.ipynb) - Fine-tune and evaluate the Llama 3.2 multi-modal model. This notebook covers creating and monitoring a fine-tuning job, visualizing training metrics, setting up provisioned throughput, running inference with the fine-tuned model on test images, and cleaning up AWS resources. 10 | 11 | ## Prerequisites 12 | 13 | - An AWS account with access to Amazon Bedrock 14 | - Appropriate IAM permissions for Bedrock, S3, and IAM role creation 15 | - Python 3.8+ environment with required libraries 16 | 17 | ## Important Notes 18 | - Llama 3.2 fine-tuning is currently only available in the us-west-2 AWS region 19 | - Fine-tuning jobs may take several hours to complete depending on dataset size 20 | - The COCO image dataset is approximately 19.3 GB in size and requires at least 25 GB of free disk space 21 | 22 | 23 | ## Contributing 24 | 25 | We welcome community contributions! Please ensure your sample aligns with AWS [best practices](https://aws.amazon.com/architecture/well-architected/), and please update the **Contents** section of this README file with a link to your sample, along with a description. -------------------------------------------------------------------------------- /03_Model_customization/continued Pre-training/03_cleanup.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "15a45d36-41d9-4635-9650-d456d25bd33c", 6 | "metadata": {}, 7 | "source": [ 8 | "## Clean up\n", 9 | "\n", 10 | "Its important to clean up all the resources, that you have created in the previous notebooks as there might be cost associated with the resources. \n", 11 | "> *This notebook should work well with the **`Data Science 3.0`**, **`Python 3`**, and **`ml.t3.medium`** kernel in SageMaker Studio*\n" 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "id": "3bb1ff61-faf4-491d-a194-6ba9bd99f64a", 17 | "metadata": {}, 18 | "source": [ 19 | "## Retrieve resource names for deleting " 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "id": "2b6a6b96-1939-4214-a1e9-5b724e281247", 26 | "metadata": { 27 | "tags": [] 28 | }, 29 | "outputs": [], 30 | "source": [ 31 | "%store -r bucket_name\n", 32 | "%store -r role_name\n", 33 | "%store -r role_arn\n", 34 | "%store -r policy_arn" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": null, 40 | "id": "6ecb82e1-8b6e-46bb-b4d0-00b46b06c144", 41 | "metadata": { 42 | "tags": [] 43 | }, 44 | "outputs": [], 45 | "source": [ 46 | "print(bucket_name)\n", 47 | "print(role_name)\n", 48 | "print(role_arn)\n", 49 | "print(policy_arn)" 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": null, 55 | "id": "550ca2f0-7cd3-4e71-baa6-fd59ed4420fe", 56 | "metadata": { 57 | "tags": [] 58 | }, 59 | "outputs": [], 60 | "source": [ 61 | "import boto3\n", 62 | "session = boto3.session.Session()\n", 63 | "region = session.region_name\n", 64 | "s3_client = boto3.client('s3')\n", 65 | "iam = boto3.client('iam', region_name=region)" 66 | ] 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "id": "19a9d198-d1d5-4b6a-bcdb-3df46fd58156", 71 | "metadata": {}, 72 | "source": [ 73 | "## Delete S3 bucket" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "id": "29789daa-b5c0-48ff-ae1e-c4c8a425f7ab", 80 | "metadata": { 81 | "tags": [] 82 | }, 83 | "outputs": [], 84 | "source": [ 85 | "objects = s3_client.list_objects(Bucket=bucket_name) \n", 86 | "if 'Contents' in objects:\n", 87 | " for obj in objects['Contents']:\n", 88 | " s3_client.delete_object(Bucket=bucket_name, Key=obj['Key']) \n", 89 | "s3_client.delete_bucket(Bucket=bucket_name)" 90 | ] 91 | }, 92 | { 93 | "cell_type": "markdown", 94 | "id": "32aa1b75-38ca-41c8-9587-15718fb1d527", 95 | "metadata": {}, 96 | "source": [ 97 | "## Delete roles and policies" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": null, 103 | "id": "5cb29ce2-bf21-48b6-8b71-1274d4cea058", 104 | "metadata": { 105 | "tags": [] 106 | }, 107 | "outputs": [], 108 | "source": [ 109 | "iam.detach_role_policy(RoleName=role_name, PolicyArn=policy_arn)\n", 110 | "iam.delete_role(RoleName=role_name)" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": null, 116 | "id": "91ccfb15-a6f6-409b-9e8e-7350e2c15c8f", 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [] 120 | } 121 | ], 122 | "metadata": { 123 | "availableInstances": [ 124 | { 125 | "_defaultOrder": 0, 126 | "_isFastLaunch": true, 127 | "category": "General purpose", 128 | "gpuNum": 0, 129 | "hideHardwareSpecs": false, 130 | "memoryGiB": 4, 131 | "name": "ml.t3.medium", 132 | "vcpuNum": 2 133 | }, 134 | { 135 | "_defaultOrder": 1, 136 | "_isFastLaunch": false, 137 | "category": "General purpose", 138 | "gpuNum": 0, 139 | "hideHardwareSpecs": false, 140 | "memoryGiB": 8, 141 | "name": "ml.t3.large", 142 | "vcpuNum": 2 143 | }, 144 | { 145 | "_defaultOrder": 2, 146 | "_isFastLaunch": false, 147 | "category": "General purpose", 148 | "gpuNum": 0, 149 | "hideHardwareSpecs": false, 150 | "memoryGiB": 16, 151 | "name": "ml.t3.xlarge", 152 | "vcpuNum": 4 153 | }, 154 | { 155 | "_defaultOrder": 3, 156 | "_isFastLaunch": false, 157 | "category": "General purpose", 158 | "gpuNum": 0, 159 | "hideHardwareSpecs": false, 160 | "memoryGiB": 32, 161 | "name": "ml.t3.2xlarge", 162 | "vcpuNum": 8 163 | }, 164 | { 165 | "_defaultOrder": 4, 166 | "_isFastLaunch": true, 167 | "category": "General purpose", 168 | "gpuNum": 0, 169 | "hideHardwareSpecs": false, 170 | "memoryGiB": 8, 171 | "name": "ml.m5.large", 172 | "vcpuNum": 2 173 | }, 174 | { 175 | "_defaultOrder": 5, 176 | "_isFastLaunch": false, 177 | "category": "General purpose", 178 | "gpuNum": 0, 179 | "hideHardwareSpecs": false, 180 | "memoryGiB": 16, 181 | "name": "ml.m5.xlarge", 182 | "vcpuNum": 4 183 | }, 184 | { 185 | "_defaultOrder": 6, 186 | "_isFastLaunch": false, 187 | "category": "General purpose", 188 | "gpuNum": 0, 189 | "hideHardwareSpecs": false, 190 | "memoryGiB": 32, 191 | "name": "ml.m5.2xlarge", 192 | "vcpuNum": 8 193 | }, 194 | { 195 | "_defaultOrder": 7, 196 | "_isFastLaunch": false, 197 | "category": "General purpose", 198 | "gpuNum": 0, 199 | "hideHardwareSpecs": false, 200 | "memoryGiB": 64, 201 | "name": "ml.m5.4xlarge", 202 | "vcpuNum": 16 203 | }, 204 | { 205 | "_defaultOrder": 8, 206 | "_isFastLaunch": false, 207 | "category": "General purpose", 208 | "gpuNum": 0, 209 | "hideHardwareSpecs": false, 210 | "memoryGiB": 128, 211 | "name": "ml.m5.8xlarge", 212 | "vcpuNum": 32 213 | }, 214 | { 215 | "_defaultOrder": 9, 216 | "_isFastLaunch": false, 217 | "category": "General purpose", 218 | "gpuNum": 0, 219 | "hideHardwareSpecs": false, 220 | "memoryGiB": 192, 221 | "name": "ml.m5.12xlarge", 222 | "vcpuNum": 48 223 | }, 224 | { 225 | "_defaultOrder": 10, 226 | "_isFastLaunch": false, 227 | "category": "General purpose", 228 | "gpuNum": 0, 229 | "hideHardwareSpecs": false, 230 | "memoryGiB": 256, 231 | "name": "ml.m5.16xlarge", 232 | "vcpuNum": 64 233 | }, 234 | { 235 | "_defaultOrder": 11, 236 | "_isFastLaunch": false, 237 | "category": "General purpose", 238 | "gpuNum": 0, 239 | "hideHardwareSpecs": false, 240 | "memoryGiB": 384, 241 | "name": "ml.m5.24xlarge", 242 | "vcpuNum": 96 243 | }, 244 | { 245 | "_defaultOrder": 12, 246 | "_isFastLaunch": false, 247 | "category": "General purpose", 248 | "gpuNum": 0, 249 | "hideHardwareSpecs": false, 250 | "memoryGiB": 8, 251 | "name": "ml.m5d.large", 252 | "vcpuNum": 2 253 | }, 254 | { 255 | "_defaultOrder": 13, 256 | "_isFastLaunch": false, 257 | "category": "General purpose", 258 | "gpuNum": 0, 259 | "hideHardwareSpecs": false, 260 | "memoryGiB": 16, 261 | "name": "ml.m5d.xlarge", 262 | "vcpuNum": 4 263 | }, 264 | { 265 | "_defaultOrder": 14, 266 | "_isFastLaunch": false, 267 | "category": "General purpose", 268 | "gpuNum": 0, 269 | "hideHardwareSpecs": false, 270 | "memoryGiB": 32, 271 | "name": "ml.m5d.2xlarge", 272 | "vcpuNum": 8 273 | }, 274 | { 275 | "_defaultOrder": 15, 276 | "_isFastLaunch": false, 277 | "category": "General purpose", 278 | "gpuNum": 0, 279 | "hideHardwareSpecs": false, 280 | "memoryGiB": 64, 281 | "name": "ml.m5d.4xlarge", 282 | "vcpuNum": 16 283 | }, 284 | { 285 | "_defaultOrder": 16, 286 | "_isFastLaunch": false, 287 | "category": "General purpose", 288 | "gpuNum": 0, 289 | "hideHardwareSpecs": false, 290 | "memoryGiB": 128, 291 | "name": "ml.m5d.8xlarge", 292 | "vcpuNum": 32 293 | }, 294 | { 295 | "_defaultOrder": 17, 296 | "_isFastLaunch": false, 297 | "category": "General purpose", 298 | "gpuNum": 0, 299 | "hideHardwareSpecs": false, 300 | "memoryGiB": 192, 301 | "name": "ml.m5d.12xlarge", 302 | "vcpuNum": 48 303 | }, 304 | { 305 | "_defaultOrder": 18, 306 | "_isFastLaunch": false, 307 | "category": "General purpose", 308 | "gpuNum": 0, 309 | "hideHardwareSpecs": false, 310 | "memoryGiB": 256, 311 | "name": "ml.m5d.16xlarge", 312 | "vcpuNum": 64 313 | }, 314 | { 315 | "_defaultOrder": 19, 316 | "_isFastLaunch": false, 317 | "category": "General purpose", 318 | "gpuNum": 0, 319 | "hideHardwareSpecs": false, 320 | "memoryGiB": 384, 321 | "name": "ml.m5d.24xlarge", 322 | "vcpuNum": 96 323 | }, 324 | { 325 | "_defaultOrder": 20, 326 | "_isFastLaunch": false, 327 | "category": "General purpose", 328 | "gpuNum": 0, 329 | "hideHardwareSpecs": true, 330 | "memoryGiB": 0, 331 | "name": "ml.geospatial.interactive", 332 | "supportedImageNames": [ 333 | "sagemaker-geospatial-v1-0" 334 | ], 335 | "vcpuNum": 0 336 | }, 337 | { 338 | "_defaultOrder": 21, 339 | "_isFastLaunch": true, 340 | "category": "Compute optimized", 341 | "gpuNum": 0, 342 | "hideHardwareSpecs": false, 343 | "memoryGiB": 4, 344 | "name": "ml.c5.large", 345 | "vcpuNum": 2 346 | }, 347 | { 348 | "_defaultOrder": 22, 349 | "_isFastLaunch": false, 350 | "category": "Compute optimized", 351 | "gpuNum": 0, 352 | "hideHardwareSpecs": false, 353 | "memoryGiB": 8, 354 | "name": "ml.c5.xlarge", 355 | "vcpuNum": 4 356 | }, 357 | { 358 | "_defaultOrder": 23, 359 | "_isFastLaunch": false, 360 | "category": "Compute optimized", 361 | "gpuNum": 0, 362 | "hideHardwareSpecs": false, 363 | "memoryGiB": 16, 364 | "name": "ml.c5.2xlarge", 365 | "vcpuNum": 8 366 | }, 367 | { 368 | "_defaultOrder": 24, 369 | "_isFastLaunch": false, 370 | "category": "Compute optimized", 371 | "gpuNum": 0, 372 | "hideHardwareSpecs": false, 373 | "memoryGiB": 32, 374 | "name": "ml.c5.4xlarge", 375 | "vcpuNum": 16 376 | }, 377 | { 378 | "_defaultOrder": 25, 379 | "_isFastLaunch": false, 380 | "category": "Compute optimized", 381 | "gpuNum": 0, 382 | "hideHardwareSpecs": false, 383 | "memoryGiB": 72, 384 | "name": "ml.c5.9xlarge", 385 | "vcpuNum": 36 386 | }, 387 | { 388 | "_defaultOrder": 26, 389 | "_isFastLaunch": false, 390 | "category": "Compute optimized", 391 | "gpuNum": 0, 392 | "hideHardwareSpecs": false, 393 | "memoryGiB": 96, 394 | "name": "ml.c5.12xlarge", 395 | "vcpuNum": 48 396 | }, 397 | { 398 | "_defaultOrder": 27, 399 | "_isFastLaunch": false, 400 | "category": "Compute optimized", 401 | "gpuNum": 0, 402 | "hideHardwareSpecs": false, 403 | "memoryGiB": 144, 404 | "name": "ml.c5.18xlarge", 405 | "vcpuNum": 72 406 | }, 407 | { 408 | "_defaultOrder": 28, 409 | "_isFastLaunch": false, 410 | "category": "Compute optimized", 411 | "gpuNum": 0, 412 | "hideHardwareSpecs": false, 413 | "memoryGiB": 192, 414 | "name": "ml.c5.24xlarge", 415 | "vcpuNum": 96 416 | }, 417 | { 418 | "_defaultOrder": 29, 419 | "_isFastLaunch": true, 420 | "category": "Accelerated computing", 421 | "gpuNum": 1, 422 | "hideHardwareSpecs": false, 423 | "memoryGiB": 16, 424 | "name": "ml.g4dn.xlarge", 425 | "vcpuNum": 4 426 | }, 427 | { 428 | "_defaultOrder": 30, 429 | "_isFastLaunch": false, 430 | "category": "Accelerated computing", 431 | "gpuNum": 1, 432 | "hideHardwareSpecs": false, 433 | "memoryGiB": 32, 434 | "name": "ml.g4dn.2xlarge", 435 | "vcpuNum": 8 436 | }, 437 | { 438 | "_defaultOrder": 31, 439 | "_isFastLaunch": false, 440 | "category": "Accelerated computing", 441 | "gpuNum": 1, 442 | "hideHardwareSpecs": false, 443 | "memoryGiB": 64, 444 | "name": "ml.g4dn.4xlarge", 445 | "vcpuNum": 16 446 | }, 447 | { 448 | "_defaultOrder": 32, 449 | "_isFastLaunch": false, 450 | "category": "Accelerated computing", 451 | "gpuNum": 1, 452 | "hideHardwareSpecs": false, 453 | "memoryGiB": 128, 454 | "name": "ml.g4dn.8xlarge", 455 | "vcpuNum": 32 456 | }, 457 | { 458 | "_defaultOrder": 33, 459 | "_isFastLaunch": false, 460 | "category": "Accelerated computing", 461 | "gpuNum": 4, 462 | "hideHardwareSpecs": false, 463 | "memoryGiB": 192, 464 | "name": "ml.g4dn.12xlarge", 465 | "vcpuNum": 48 466 | }, 467 | { 468 | "_defaultOrder": 34, 469 | "_isFastLaunch": false, 470 | "category": "Accelerated computing", 471 | "gpuNum": 1, 472 | "hideHardwareSpecs": false, 473 | "memoryGiB": 256, 474 | "name": "ml.g4dn.16xlarge", 475 | "vcpuNum": 64 476 | }, 477 | { 478 | "_defaultOrder": 35, 479 | "_isFastLaunch": false, 480 | "category": "Accelerated computing", 481 | "gpuNum": 1, 482 | "hideHardwareSpecs": false, 483 | "memoryGiB": 61, 484 | "name": "ml.p3.2xlarge", 485 | "vcpuNum": 8 486 | }, 487 | { 488 | "_defaultOrder": 36, 489 | "_isFastLaunch": false, 490 | "category": "Accelerated computing", 491 | "gpuNum": 4, 492 | "hideHardwareSpecs": false, 493 | "memoryGiB": 244, 494 | "name": "ml.p3.8xlarge", 495 | "vcpuNum": 32 496 | }, 497 | { 498 | "_defaultOrder": 37, 499 | "_isFastLaunch": false, 500 | "category": "Accelerated computing", 501 | "gpuNum": 8, 502 | "hideHardwareSpecs": false, 503 | "memoryGiB": 488, 504 | "name": "ml.p3.16xlarge", 505 | "vcpuNum": 64 506 | }, 507 | { 508 | "_defaultOrder": 38, 509 | "_isFastLaunch": false, 510 | "category": "Accelerated computing", 511 | "gpuNum": 8, 512 | "hideHardwareSpecs": false, 513 | "memoryGiB": 768, 514 | "name": "ml.p3dn.24xlarge", 515 | "vcpuNum": 96 516 | }, 517 | { 518 | "_defaultOrder": 39, 519 | "_isFastLaunch": false, 520 | "category": "Memory Optimized", 521 | "gpuNum": 0, 522 | "hideHardwareSpecs": false, 523 | "memoryGiB": 16, 524 | "name": "ml.r5.large", 525 | "vcpuNum": 2 526 | }, 527 | { 528 | "_defaultOrder": 40, 529 | "_isFastLaunch": false, 530 | "category": "Memory Optimized", 531 | "gpuNum": 0, 532 | "hideHardwareSpecs": false, 533 | "memoryGiB": 32, 534 | "name": "ml.r5.xlarge", 535 | "vcpuNum": 4 536 | }, 537 | { 538 | "_defaultOrder": 41, 539 | "_isFastLaunch": false, 540 | "category": "Memory Optimized", 541 | "gpuNum": 0, 542 | "hideHardwareSpecs": false, 543 | "memoryGiB": 64, 544 | "name": "ml.r5.2xlarge", 545 | "vcpuNum": 8 546 | }, 547 | { 548 | "_defaultOrder": 42, 549 | "_isFastLaunch": false, 550 | "category": "Memory Optimized", 551 | "gpuNum": 0, 552 | "hideHardwareSpecs": false, 553 | "memoryGiB": 128, 554 | "name": "ml.r5.4xlarge", 555 | "vcpuNum": 16 556 | }, 557 | { 558 | "_defaultOrder": 43, 559 | "_isFastLaunch": false, 560 | "category": "Memory Optimized", 561 | "gpuNum": 0, 562 | "hideHardwareSpecs": false, 563 | "memoryGiB": 256, 564 | "name": "ml.r5.8xlarge", 565 | "vcpuNum": 32 566 | }, 567 | { 568 | "_defaultOrder": 44, 569 | "_isFastLaunch": false, 570 | "category": "Memory Optimized", 571 | "gpuNum": 0, 572 | "hideHardwareSpecs": false, 573 | "memoryGiB": 384, 574 | "name": "ml.r5.12xlarge", 575 | "vcpuNum": 48 576 | }, 577 | { 578 | "_defaultOrder": 45, 579 | "_isFastLaunch": false, 580 | "category": "Memory Optimized", 581 | "gpuNum": 0, 582 | "hideHardwareSpecs": false, 583 | "memoryGiB": 512, 584 | "name": "ml.r5.16xlarge", 585 | "vcpuNum": 64 586 | }, 587 | { 588 | "_defaultOrder": 46, 589 | "_isFastLaunch": false, 590 | "category": "Memory Optimized", 591 | "gpuNum": 0, 592 | "hideHardwareSpecs": false, 593 | "memoryGiB": 768, 594 | "name": "ml.r5.24xlarge", 595 | "vcpuNum": 96 596 | }, 597 | { 598 | "_defaultOrder": 47, 599 | "_isFastLaunch": false, 600 | "category": "Accelerated computing", 601 | "gpuNum": 1, 602 | "hideHardwareSpecs": false, 603 | "memoryGiB": 16, 604 | "name": "ml.g5.xlarge", 605 | "vcpuNum": 4 606 | }, 607 | { 608 | "_defaultOrder": 48, 609 | "_isFastLaunch": false, 610 | "category": "Accelerated computing", 611 | "gpuNum": 1, 612 | "hideHardwareSpecs": false, 613 | "memoryGiB": 32, 614 | "name": "ml.g5.2xlarge", 615 | "vcpuNum": 8 616 | }, 617 | { 618 | "_defaultOrder": 49, 619 | "_isFastLaunch": false, 620 | "category": "Accelerated computing", 621 | "gpuNum": 1, 622 | "hideHardwareSpecs": false, 623 | "memoryGiB": 64, 624 | "name": "ml.g5.4xlarge", 625 | "vcpuNum": 16 626 | }, 627 | { 628 | "_defaultOrder": 50, 629 | "_isFastLaunch": false, 630 | "category": "Accelerated computing", 631 | "gpuNum": 1, 632 | "hideHardwareSpecs": false, 633 | "memoryGiB": 128, 634 | "name": "ml.g5.8xlarge", 635 | "vcpuNum": 32 636 | }, 637 | { 638 | "_defaultOrder": 51, 639 | "_isFastLaunch": false, 640 | "category": "Accelerated computing", 641 | "gpuNum": 1, 642 | "hideHardwareSpecs": false, 643 | "memoryGiB": 256, 644 | "name": "ml.g5.16xlarge", 645 | "vcpuNum": 64 646 | }, 647 | { 648 | "_defaultOrder": 52, 649 | "_isFastLaunch": false, 650 | "category": "Accelerated computing", 651 | "gpuNum": 4, 652 | "hideHardwareSpecs": false, 653 | "memoryGiB": 192, 654 | "name": "ml.g5.12xlarge", 655 | "vcpuNum": 48 656 | }, 657 | { 658 | "_defaultOrder": 53, 659 | "_isFastLaunch": false, 660 | "category": "Accelerated computing", 661 | "gpuNum": 4, 662 | "hideHardwareSpecs": false, 663 | "memoryGiB": 384, 664 | "name": "ml.g5.24xlarge", 665 | "vcpuNum": 96 666 | }, 667 | { 668 | "_defaultOrder": 54, 669 | "_isFastLaunch": false, 670 | "category": "Accelerated computing", 671 | "gpuNum": 8, 672 | "hideHardwareSpecs": false, 673 | "memoryGiB": 768, 674 | "name": "ml.g5.48xlarge", 675 | "vcpuNum": 192 676 | }, 677 | { 678 | "_defaultOrder": 55, 679 | "_isFastLaunch": false, 680 | "category": "Accelerated computing", 681 | "gpuNum": 8, 682 | "hideHardwareSpecs": false, 683 | "memoryGiB": 1152, 684 | "name": "ml.p4d.24xlarge", 685 | "vcpuNum": 96 686 | }, 687 | { 688 | "_defaultOrder": 56, 689 | "_isFastLaunch": false, 690 | "category": "Accelerated computing", 691 | "gpuNum": 8, 692 | "hideHardwareSpecs": false, 693 | "memoryGiB": 1152, 694 | "name": "ml.p4de.24xlarge", 695 | "vcpuNum": 96 696 | }, 697 | { 698 | "_defaultOrder": 57, 699 | "_isFastLaunch": false, 700 | "category": "Accelerated computing", 701 | "gpuNum": 0, 702 | "hideHardwareSpecs": false, 703 | "memoryGiB": 32, 704 | "name": "ml.trn1.2xlarge", 705 | "vcpuNum": 8 706 | }, 707 | { 708 | "_defaultOrder": 58, 709 | "_isFastLaunch": false, 710 | "category": "Accelerated computing", 711 | "gpuNum": 0, 712 | "hideHardwareSpecs": false, 713 | "memoryGiB": 512, 714 | "name": "ml.trn1.32xlarge", 715 | "vcpuNum": 128 716 | }, 717 | { 718 | "_defaultOrder": 59, 719 | "_isFastLaunch": false, 720 | "category": "Accelerated computing", 721 | "gpuNum": 0, 722 | "hideHardwareSpecs": false, 723 | "memoryGiB": 512, 724 | "name": "ml.trn1n.32xlarge", 725 | "vcpuNum": 128 726 | } 727 | ], 728 | "instance_type": "ml.t3.medium", 729 | "kernelspec": { 730 | "display_name": "Python 3 (Data Science 3.0)", 731 | "language": "python", 732 | "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1" 733 | }, 734 | "language_info": { 735 | "codemirror_mode": { 736 | "name": "ipython", 737 | "version": 3 738 | }, 739 | "file_extension": ".py", 740 | "mimetype": "text/x-python", 741 | "name": "python", 742 | "nbconvert_exporter": "python", 743 | "pygments_lexer": "ipython3", 744 | "version": "3.10.6" 745 | } 746 | }, 747 | "nbformat": 4, 748 | "nbformat_minor": 5 749 | } 750 | -------------------------------------------------------------------------------- /04_Image_and_Multimodal/README.md: -------------------------------------------------------------------------------- 1 | # Module 4 - Image Generation and Multimodal Embeddings 2 | 3 | ## Overview 4 | 5 | Image generation can be a tedious task for artists, designers and content creators who illustrate their thoughts with the help of images. With the help of Foundation Models (FMs) this tedious task can be streamlined to just a single line of text that expresses the thoughts of the artist, FMs can be used for creating realistic and artistic images of various subjects, environments, and scenes from language prompts. 6 | 7 | Image indexing and searching is another tedious enterprise task. With the help of FMs, enterprise can build multimodal image indexing, searching and recommendation applications quickly. 8 | 9 | In this lab, we will explore how to use FMs available in Amazon Bedrock to generate images as well as modify existing images, and how to use FMs to do multimodal image indexing and searching. 10 | 11 | 12 | ## Prompt Engineering for Images 13 | 14 | Writing a good prompt can sometimes be an art. It is often difficult to predict whether a certain prompt will yield a satisfactory image with a given model. However, there are certain templates that have been observed to work. Broadly speaking, a prompt can be roughly broken down into three pieces: 15 | 16 | * type of image (photograph/sketch/painting etc.), and 17 | * description (subject/object/environment/scene etc.), and 18 | * the style of the image (realistic/artistic/type of art etc.). 19 | 20 | You can change each of the three parts individually, to generate variations of an image. Adjectives have been known to play a significant role in the image generation process. Also, adding more details help in the generation process.To generate a realistic image, you can use phrases such as "a photo of", "a photograph of", "realistic" or "hyper realistic". 21 | 22 | To generate images by artists you can use phrases like "by Pablo Picasso" or "oil painting by Rembrandt" or "landscape art by Frederic Edwin Church" or "pencil drawing by Albrecht Dürer". You can also combine different artists as well. To generate artistic image by category, you can add the art category in the prompt such as "lion on a beach, abstract". Some other categories include "oil painting", "pencil drawing", "pop art", "digital art", "anime", "cartoon", "futurism", "watercolor", "manga" etc. You can also include details such as lighting or camera lens, such as 35mm wide lens or 85mm wide lens and details about the framing (portrait/landscape/close up etc.). 23 | 24 | Note that the model generates different images even if same prompt is given multiple times. So, you can generate multiple images and select the image that suits your application best. 25 | 26 | ## Foundation Models 27 | 28 | To provide these capabilities, Amazon Bedrock supports [Stable Diffusion XL](https://stability.ai/stablediffusion) from Stability AI and [Titan Image Generator](https://aws.amazon.com/bedrock/titan/) from Amazon for image generation, and [Titan Multimodal Embeddings](https://aws.amazon.com/bedrock/titan/) for multimodal image indexing and searching. 29 | 30 | ### Stable Diffusion 31 | 32 | Stable Diffusion works on the principle of diffusion and is composed of multiple models each having different purpose: 33 | 34 | 1. The CLIP text encoder; 35 | 2. The VAE decoder; 36 | 3. The UNet, and 37 | 4. The VAE_post_quant_conv 38 | 39 | The workings can be explained with this architecture: 40 | ![Stable Diffusion Architecture](./images/sd.png) 41 | 42 | ### Titan Image Generator 43 | 44 | Titan Image Generator G1 is an image generation model. It generates images from text, and allows users to upload and edit an existing image. Users can edit an image with a text prompt (without a mask) or parts of an image with an image mask, or extend the boundaries of an image with outpainting. It can also generate variations of an image. 45 | 46 | ### Titan Multimodal Embeddings 47 | 48 | Titan Multimodal Embeddings Generation 1 (G1) is a multimodal embeddings model for use cases like searching images by text, image, or a combination of text and image. Designed for high accuracy and fast responses, this model is an ideal choice for search and recommendations use cases. 49 | 50 | ## Target Audience 51 | 52 | Marketing companies, agencies, web-designers, and general companies can take advantage of this feature to generate brand new images, from scratch. 53 | -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/71-txt-2-img.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/71-txt-2-img.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/72-img-2-img.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/72-img-2-img.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/generated_dog.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/generated_dog.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/mask.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/multimodal-embeddings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/multimodal-embeddings.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/octank_color_palette.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/octank_color_palette.JPG -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/sd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/sd.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/sketch_dog.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/sketch_dog.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/three_pots-add_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/three_pots-add_mask.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/three_pots-add_mask_INVERTED.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/three_pots-add_mask_INVERTED.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/three_pots-center_pot_mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/three_pots-center_pot_mask.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/three_pots-center_pot_mask_INVERTED.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/three_pots-center_pot_mask_INVERTED.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/three_pots.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/three_pots.jpg -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/titan_image_generator_playground.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/titan_image_generator_playground.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/video_input.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/video_input.png -------------------------------------------------------------------------------- /04_Image_and_Multimodal/images/woman-in-gallery.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/04_Image_and_Multimodal/images/woman-in-gallery.jpg -------------------------------------------------------------------------------- /04_Image_and_Multimodal/requirements.txt: -------------------------------------------------------------------------------- 1 | seaborn 2 | numpy -------------------------------------------------------------------------------- /05_Agents/04_clean_up_agent_resources.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Agents for Amazon Bedrock - Cleaning up resources\n", 8 | "\n", 9 | "Do not run this notebook if you are planning to use the agent and knowledge base that you created.\n", 10 | "If you only wanted to try out creating the Agent and Knowledge Base using SDK, then please make sure to delete all the resources that were created as you will be incurred cost for storing documents in OSS index and in the DynamoDB table." 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": null, 16 | "metadata": { 17 | "pycharm": { 18 | "name": "#%%\n" 19 | } 20 | }, 21 | "outputs": [], 22 | "source": [ 23 | "%store -r" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "### Importing the support functions" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": { 37 | "pycharm": { 38 | "name": "#%%\n" 39 | } 40 | }, 41 | "outputs": [], 42 | "source": [ 43 | "from agent import delete_agent_roles_and_policies, clean_up_resources\n", 44 | "from knowledge_base import KnowledgeBasesForAmazonBedrock" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "Cleaning up agent's resources" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": null, 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "clean_up_resources(\n", 61 | " table_name, lambda_function, lambda_function_name, agent_action_group_response, agent_functions, \n", 62 | " agent_id, kb_id, alias_id\n", 63 | ")" 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "Deleting Agent's roles and policies" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "# Delete the agent roles and policies\n", 80 | "delete_agent_roles_and_policies(agent_name, kb_policy_name)" 81 | ] 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "metadata": {}, 86 | "source": [ 87 | "Deleting Knowledge Base" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": null, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "kb = KnowledgeBasesForAmazonBedrock()\n", 97 | "kb.delete_kb(knowledge_base_name, delete_s3_bucket=True, delete_iam_roles_and_policies=True)" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "### Next Steps\n", 105 | "\n", 106 | "Next we will discuss some Open Source examples" 107 | ] 108 | } 109 | ], 110 | "metadata": { 111 | "availableInstances": [ 112 | { 113 | "_defaultOrder": 0, 114 | "_isFastLaunch": true, 115 | "category": "General purpose", 116 | "gpuNum": 0, 117 | "hideHardwareSpecs": false, 118 | "memoryGiB": 4, 119 | "name": "ml.t3.medium", 120 | "vcpuNum": 2 121 | }, 122 | { 123 | "_defaultOrder": 1, 124 | "_isFastLaunch": false, 125 | "category": "General purpose", 126 | "gpuNum": 0, 127 | "hideHardwareSpecs": false, 128 | "memoryGiB": 8, 129 | "name": "ml.t3.large", 130 | "vcpuNum": 2 131 | }, 132 | { 133 | "_defaultOrder": 2, 134 | "_isFastLaunch": false, 135 | "category": "General purpose", 136 | "gpuNum": 0, 137 | "hideHardwareSpecs": false, 138 | "memoryGiB": 16, 139 | "name": "ml.t3.xlarge", 140 | "vcpuNum": 4 141 | }, 142 | { 143 | "_defaultOrder": 3, 144 | "_isFastLaunch": false, 145 | "category": "General purpose", 146 | "gpuNum": 0, 147 | "hideHardwareSpecs": false, 148 | "memoryGiB": 32, 149 | "name": "ml.t3.2xlarge", 150 | "vcpuNum": 8 151 | }, 152 | { 153 | "_defaultOrder": 4, 154 | "_isFastLaunch": true, 155 | "category": "General purpose", 156 | "gpuNum": 0, 157 | "hideHardwareSpecs": false, 158 | "memoryGiB": 8, 159 | "name": "ml.m5.large", 160 | "vcpuNum": 2 161 | }, 162 | { 163 | "_defaultOrder": 5, 164 | "_isFastLaunch": false, 165 | "category": "General purpose", 166 | "gpuNum": 0, 167 | "hideHardwareSpecs": false, 168 | "memoryGiB": 16, 169 | "name": "ml.m5.xlarge", 170 | "vcpuNum": 4 171 | }, 172 | { 173 | "_defaultOrder": 6, 174 | "_isFastLaunch": false, 175 | "category": "General purpose", 176 | "gpuNum": 0, 177 | "hideHardwareSpecs": false, 178 | "memoryGiB": 32, 179 | "name": "ml.m5.2xlarge", 180 | "vcpuNum": 8 181 | }, 182 | { 183 | "_defaultOrder": 7, 184 | "_isFastLaunch": false, 185 | "category": "General purpose", 186 | "gpuNum": 0, 187 | "hideHardwareSpecs": false, 188 | "memoryGiB": 64, 189 | "name": "ml.m5.4xlarge", 190 | "vcpuNum": 16 191 | }, 192 | { 193 | "_defaultOrder": 8, 194 | "_isFastLaunch": false, 195 | "category": "General purpose", 196 | "gpuNum": 0, 197 | "hideHardwareSpecs": false, 198 | "memoryGiB": 128, 199 | "name": "ml.m5.8xlarge", 200 | "vcpuNum": 32 201 | }, 202 | { 203 | "_defaultOrder": 9, 204 | "_isFastLaunch": false, 205 | "category": "General purpose", 206 | "gpuNum": 0, 207 | "hideHardwareSpecs": false, 208 | "memoryGiB": 192, 209 | "name": "ml.m5.12xlarge", 210 | "vcpuNum": 48 211 | }, 212 | { 213 | "_defaultOrder": 10, 214 | "_isFastLaunch": false, 215 | "category": "General purpose", 216 | "gpuNum": 0, 217 | "hideHardwareSpecs": false, 218 | "memoryGiB": 256, 219 | "name": "ml.m5.16xlarge", 220 | "vcpuNum": 64 221 | }, 222 | { 223 | "_defaultOrder": 11, 224 | "_isFastLaunch": false, 225 | "category": "General purpose", 226 | "gpuNum": 0, 227 | "hideHardwareSpecs": false, 228 | "memoryGiB": 384, 229 | "name": "ml.m5.24xlarge", 230 | "vcpuNum": 96 231 | }, 232 | { 233 | "_defaultOrder": 12, 234 | "_isFastLaunch": false, 235 | "category": "General purpose", 236 | "gpuNum": 0, 237 | "hideHardwareSpecs": false, 238 | "memoryGiB": 8, 239 | "name": "ml.m5d.large", 240 | "vcpuNum": 2 241 | }, 242 | { 243 | "_defaultOrder": 13, 244 | "_isFastLaunch": false, 245 | "category": "General purpose", 246 | "gpuNum": 0, 247 | "hideHardwareSpecs": false, 248 | "memoryGiB": 16, 249 | "name": "ml.m5d.xlarge", 250 | "vcpuNum": 4 251 | }, 252 | { 253 | "_defaultOrder": 14, 254 | "_isFastLaunch": false, 255 | "category": "General purpose", 256 | "gpuNum": 0, 257 | "hideHardwareSpecs": false, 258 | "memoryGiB": 32, 259 | "name": "ml.m5d.2xlarge", 260 | "vcpuNum": 8 261 | }, 262 | { 263 | "_defaultOrder": 15, 264 | "_isFastLaunch": false, 265 | "category": "General purpose", 266 | "gpuNum": 0, 267 | "hideHardwareSpecs": false, 268 | "memoryGiB": 64, 269 | "name": "ml.m5d.4xlarge", 270 | "vcpuNum": 16 271 | }, 272 | { 273 | "_defaultOrder": 16, 274 | "_isFastLaunch": false, 275 | "category": "General purpose", 276 | "gpuNum": 0, 277 | "hideHardwareSpecs": false, 278 | "memoryGiB": 128, 279 | "name": "ml.m5d.8xlarge", 280 | "vcpuNum": 32 281 | }, 282 | { 283 | "_defaultOrder": 17, 284 | "_isFastLaunch": false, 285 | "category": "General purpose", 286 | "gpuNum": 0, 287 | "hideHardwareSpecs": false, 288 | "memoryGiB": 192, 289 | "name": "ml.m5d.12xlarge", 290 | "vcpuNum": 48 291 | }, 292 | { 293 | "_defaultOrder": 18, 294 | "_isFastLaunch": false, 295 | "category": "General purpose", 296 | "gpuNum": 0, 297 | "hideHardwareSpecs": false, 298 | "memoryGiB": 256, 299 | "name": "ml.m5d.16xlarge", 300 | "vcpuNum": 64 301 | }, 302 | { 303 | "_defaultOrder": 19, 304 | "_isFastLaunch": false, 305 | "category": "General purpose", 306 | "gpuNum": 0, 307 | "hideHardwareSpecs": false, 308 | "memoryGiB": 384, 309 | "name": "ml.m5d.24xlarge", 310 | "vcpuNum": 96 311 | }, 312 | { 313 | "_defaultOrder": 20, 314 | "_isFastLaunch": false, 315 | "category": "General purpose", 316 | "gpuNum": 0, 317 | "hideHardwareSpecs": true, 318 | "memoryGiB": 0, 319 | "name": "ml.geospatial.interactive", 320 | "supportedImageNames": [ 321 | "sagemaker-geospatial-v1-0" 322 | ], 323 | "vcpuNum": 0 324 | }, 325 | { 326 | "_defaultOrder": 21, 327 | "_isFastLaunch": true, 328 | "category": "Compute optimized", 329 | "gpuNum": 0, 330 | "hideHardwareSpecs": false, 331 | "memoryGiB": 4, 332 | "name": "ml.c5.large", 333 | "vcpuNum": 2 334 | }, 335 | { 336 | "_defaultOrder": 22, 337 | "_isFastLaunch": false, 338 | "category": "Compute optimized", 339 | "gpuNum": 0, 340 | "hideHardwareSpecs": false, 341 | "memoryGiB": 8, 342 | "name": "ml.c5.xlarge", 343 | "vcpuNum": 4 344 | }, 345 | { 346 | "_defaultOrder": 23, 347 | "_isFastLaunch": false, 348 | "category": "Compute optimized", 349 | "gpuNum": 0, 350 | "hideHardwareSpecs": false, 351 | "memoryGiB": 16, 352 | "name": "ml.c5.2xlarge", 353 | "vcpuNum": 8 354 | }, 355 | { 356 | "_defaultOrder": 24, 357 | "_isFastLaunch": false, 358 | "category": "Compute optimized", 359 | "gpuNum": 0, 360 | "hideHardwareSpecs": false, 361 | "memoryGiB": 32, 362 | "name": "ml.c5.4xlarge", 363 | "vcpuNum": 16 364 | }, 365 | { 366 | "_defaultOrder": 25, 367 | "_isFastLaunch": false, 368 | "category": "Compute optimized", 369 | "gpuNum": 0, 370 | "hideHardwareSpecs": false, 371 | "memoryGiB": 72, 372 | "name": "ml.c5.9xlarge", 373 | "vcpuNum": 36 374 | }, 375 | { 376 | "_defaultOrder": 26, 377 | "_isFastLaunch": false, 378 | "category": "Compute optimized", 379 | "gpuNum": 0, 380 | "hideHardwareSpecs": false, 381 | "memoryGiB": 96, 382 | "name": "ml.c5.12xlarge", 383 | "vcpuNum": 48 384 | }, 385 | { 386 | "_defaultOrder": 27, 387 | "_isFastLaunch": false, 388 | "category": "Compute optimized", 389 | "gpuNum": 0, 390 | "hideHardwareSpecs": false, 391 | "memoryGiB": 144, 392 | "name": "ml.c5.18xlarge", 393 | "vcpuNum": 72 394 | }, 395 | { 396 | "_defaultOrder": 28, 397 | "_isFastLaunch": false, 398 | "category": "Compute optimized", 399 | "gpuNum": 0, 400 | "hideHardwareSpecs": false, 401 | "memoryGiB": 192, 402 | "name": "ml.c5.24xlarge", 403 | "vcpuNum": 96 404 | }, 405 | { 406 | "_defaultOrder": 29, 407 | "_isFastLaunch": true, 408 | "category": "Accelerated computing", 409 | "gpuNum": 1, 410 | "hideHardwareSpecs": false, 411 | "memoryGiB": 16, 412 | "name": "ml.g4dn.xlarge", 413 | "vcpuNum": 4 414 | }, 415 | { 416 | "_defaultOrder": 30, 417 | "_isFastLaunch": false, 418 | "category": "Accelerated computing", 419 | "gpuNum": 1, 420 | "hideHardwareSpecs": false, 421 | "memoryGiB": 32, 422 | "name": "ml.g4dn.2xlarge", 423 | "vcpuNum": 8 424 | }, 425 | { 426 | "_defaultOrder": 31, 427 | "_isFastLaunch": false, 428 | "category": "Accelerated computing", 429 | "gpuNum": 1, 430 | "hideHardwareSpecs": false, 431 | "memoryGiB": 64, 432 | "name": "ml.g4dn.4xlarge", 433 | "vcpuNum": 16 434 | }, 435 | { 436 | "_defaultOrder": 32, 437 | "_isFastLaunch": false, 438 | "category": "Accelerated computing", 439 | "gpuNum": 1, 440 | "hideHardwareSpecs": false, 441 | "memoryGiB": 128, 442 | "name": "ml.g4dn.8xlarge", 443 | "vcpuNum": 32 444 | }, 445 | { 446 | "_defaultOrder": 33, 447 | "_isFastLaunch": false, 448 | "category": "Accelerated computing", 449 | "gpuNum": 4, 450 | "hideHardwareSpecs": false, 451 | "memoryGiB": 192, 452 | "name": "ml.g4dn.12xlarge", 453 | "vcpuNum": 48 454 | }, 455 | { 456 | "_defaultOrder": 34, 457 | "_isFastLaunch": false, 458 | "category": "Accelerated computing", 459 | "gpuNum": 1, 460 | "hideHardwareSpecs": false, 461 | "memoryGiB": 256, 462 | "name": "ml.g4dn.16xlarge", 463 | "vcpuNum": 64 464 | }, 465 | { 466 | "_defaultOrder": 35, 467 | "_isFastLaunch": false, 468 | "category": "Accelerated computing", 469 | "gpuNum": 1, 470 | "hideHardwareSpecs": false, 471 | "memoryGiB": 61, 472 | "name": "ml.p3.2xlarge", 473 | "vcpuNum": 8 474 | }, 475 | { 476 | "_defaultOrder": 36, 477 | "_isFastLaunch": false, 478 | "category": "Accelerated computing", 479 | "gpuNum": 4, 480 | "hideHardwareSpecs": false, 481 | "memoryGiB": 244, 482 | "name": "ml.p3.8xlarge", 483 | "vcpuNum": 32 484 | }, 485 | { 486 | "_defaultOrder": 37, 487 | "_isFastLaunch": false, 488 | "category": "Accelerated computing", 489 | "gpuNum": 8, 490 | "hideHardwareSpecs": false, 491 | "memoryGiB": 488, 492 | "name": "ml.p3.16xlarge", 493 | "vcpuNum": 64 494 | }, 495 | { 496 | "_defaultOrder": 38, 497 | "_isFastLaunch": false, 498 | "category": "Accelerated computing", 499 | "gpuNum": 8, 500 | "hideHardwareSpecs": false, 501 | "memoryGiB": 768, 502 | "name": "ml.p3dn.24xlarge", 503 | "vcpuNum": 96 504 | }, 505 | { 506 | "_defaultOrder": 39, 507 | "_isFastLaunch": false, 508 | "category": "Memory Optimized", 509 | "gpuNum": 0, 510 | "hideHardwareSpecs": false, 511 | "memoryGiB": 16, 512 | "name": "ml.r5.large", 513 | "vcpuNum": 2 514 | }, 515 | { 516 | "_defaultOrder": 40, 517 | "_isFastLaunch": false, 518 | "category": "Memory Optimized", 519 | "gpuNum": 0, 520 | "hideHardwareSpecs": false, 521 | "memoryGiB": 32, 522 | "name": "ml.r5.xlarge", 523 | "vcpuNum": 4 524 | }, 525 | { 526 | "_defaultOrder": 41, 527 | "_isFastLaunch": false, 528 | "category": "Memory Optimized", 529 | "gpuNum": 0, 530 | "hideHardwareSpecs": false, 531 | "memoryGiB": 64, 532 | "name": "ml.r5.2xlarge", 533 | "vcpuNum": 8 534 | }, 535 | { 536 | "_defaultOrder": 42, 537 | "_isFastLaunch": false, 538 | "category": "Memory Optimized", 539 | "gpuNum": 0, 540 | "hideHardwareSpecs": false, 541 | "memoryGiB": 128, 542 | "name": "ml.r5.4xlarge", 543 | "vcpuNum": 16 544 | }, 545 | { 546 | "_defaultOrder": 43, 547 | "_isFastLaunch": false, 548 | "category": "Memory Optimized", 549 | "gpuNum": 0, 550 | "hideHardwareSpecs": false, 551 | "memoryGiB": 256, 552 | "name": "ml.r5.8xlarge", 553 | "vcpuNum": 32 554 | }, 555 | { 556 | "_defaultOrder": 44, 557 | "_isFastLaunch": false, 558 | "category": "Memory Optimized", 559 | "gpuNum": 0, 560 | "hideHardwareSpecs": false, 561 | "memoryGiB": 384, 562 | "name": "ml.r5.12xlarge", 563 | "vcpuNum": 48 564 | }, 565 | { 566 | "_defaultOrder": 45, 567 | "_isFastLaunch": false, 568 | "category": "Memory Optimized", 569 | "gpuNum": 0, 570 | "hideHardwareSpecs": false, 571 | "memoryGiB": 512, 572 | "name": "ml.r5.16xlarge", 573 | "vcpuNum": 64 574 | }, 575 | { 576 | "_defaultOrder": 46, 577 | "_isFastLaunch": false, 578 | "category": "Memory Optimized", 579 | "gpuNum": 0, 580 | "hideHardwareSpecs": false, 581 | "memoryGiB": 768, 582 | "name": "ml.r5.24xlarge", 583 | "vcpuNum": 96 584 | }, 585 | { 586 | "_defaultOrder": 47, 587 | "_isFastLaunch": false, 588 | "category": "Accelerated computing", 589 | "gpuNum": 1, 590 | "hideHardwareSpecs": false, 591 | "memoryGiB": 16, 592 | "name": "ml.g5.xlarge", 593 | "vcpuNum": 4 594 | }, 595 | { 596 | "_defaultOrder": 48, 597 | "_isFastLaunch": false, 598 | "category": "Accelerated computing", 599 | "gpuNum": 1, 600 | "hideHardwareSpecs": false, 601 | "memoryGiB": 32, 602 | "name": "ml.g5.2xlarge", 603 | "vcpuNum": 8 604 | }, 605 | { 606 | "_defaultOrder": 49, 607 | "_isFastLaunch": false, 608 | "category": "Accelerated computing", 609 | "gpuNum": 1, 610 | "hideHardwareSpecs": false, 611 | "memoryGiB": 64, 612 | "name": "ml.g5.4xlarge", 613 | "vcpuNum": 16 614 | }, 615 | { 616 | "_defaultOrder": 50, 617 | "_isFastLaunch": false, 618 | "category": "Accelerated computing", 619 | "gpuNum": 1, 620 | "hideHardwareSpecs": false, 621 | "memoryGiB": 128, 622 | "name": "ml.g5.8xlarge", 623 | "vcpuNum": 32 624 | }, 625 | { 626 | "_defaultOrder": 51, 627 | "_isFastLaunch": false, 628 | "category": "Accelerated computing", 629 | "gpuNum": 1, 630 | "hideHardwareSpecs": false, 631 | "memoryGiB": 256, 632 | "name": "ml.g5.16xlarge", 633 | "vcpuNum": 64 634 | }, 635 | { 636 | "_defaultOrder": 52, 637 | "_isFastLaunch": false, 638 | "category": "Accelerated computing", 639 | "gpuNum": 4, 640 | "hideHardwareSpecs": false, 641 | "memoryGiB": 192, 642 | "name": "ml.g5.12xlarge", 643 | "vcpuNum": 48 644 | }, 645 | { 646 | "_defaultOrder": 53, 647 | "_isFastLaunch": false, 648 | "category": "Accelerated computing", 649 | "gpuNum": 4, 650 | "hideHardwareSpecs": false, 651 | "memoryGiB": 384, 652 | "name": "ml.g5.24xlarge", 653 | "vcpuNum": 96 654 | }, 655 | { 656 | "_defaultOrder": 54, 657 | "_isFastLaunch": false, 658 | "category": "Accelerated computing", 659 | "gpuNum": 8, 660 | "hideHardwareSpecs": false, 661 | "memoryGiB": 768, 662 | "name": "ml.g5.48xlarge", 663 | "vcpuNum": 192 664 | }, 665 | { 666 | "_defaultOrder": 55, 667 | "_isFastLaunch": false, 668 | "category": "Accelerated computing", 669 | "gpuNum": 8, 670 | "hideHardwareSpecs": false, 671 | "memoryGiB": 1152, 672 | "name": "ml.p4d.24xlarge", 673 | "vcpuNum": 96 674 | }, 675 | { 676 | "_defaultOrder": 56, 677 | "_isFastLaunch": false, 678 | "category": "Accelerated computing", 679 | "gpuNum": 8, 680 | "hideHardwareSpecs": false, 681 | "memoryGiB": 1152, 682 | "name": "ml.p4de.24xlarge", 683 | "vcpuNum": 96 684 | }, 685 | { 686 | "_defaultOrder": 57, 687 | "_isFastLaunch": false, 688 | "category": "Accelerated computing", 689 | "gpuNum": 0, 690 | "hideHardwareSpecs": false, 691 | "memoryGiB": 32, 692 | "name": "ml.trn1.2xlarge", 693 | "vcpuNum": 8 694 | }, 695 | { 696 | "_defaultOrder": 58, 697 | "_isFastLaunch": false, 698 | "category": "Accelerated computing", 699 | "gpuNum": 0, 700 | "hideHardwareSpecs": false, 701 | "memoryGiB": 512, 702 | "name": "ml.trn1.32xlarge", 703 | "vcpuNum": 128 704 | }, 705 | { 706 | "_defaultOrder": 59, 707 | "_isFastLaunch": false, 708 | "category": "Accelerated computing", 709 | "gpuNum": 0, 710 | "hideHardwareSpecs": false, 711 | "memoryGiB": 512, 712 | "name": "ml.trn1n.32xlarge", 713 | "vcpuNum": 128 714 | } 715 | ], 716 | "instance_type": "ml.t3.medium", 717 | "kernelspec": { 718 | "display_name": "Python 3 (ipykernel)", 719 | "language": "python", 720 | "name": "python3" 721 | }, 722 | "language_info": { 723 | "codemirror_mode": { 724 | "name": "ipython", 725 | "version": 3 726 | }, 727 | "file_extension": ".py", 728 | "mimetype": "text/x-python", 729 | "name": "python", 730 | "nbconvert_exporter": "python", 731 | "pygments_lexer": "ipython3", 732 | "version": "3.11.11" 733 | } 734 | }, 735 | "nbformat": 4, 736 | "nbformat_minor": 4 737 | } 738 | -------------------------------------------------------------------------------- /05_Agents/README.md: -------------------------------------------------------------------------------- 1 | # Lab 7 - Agents for Bedrock 2 | 3 | ## Overview 4 | 5 | In this lab, you will learn about [Agents for Amazon Bedrock](https://aws.amazon.com/bedrock/agents/), 6 | an [Amazon Bedrock](https://aws.amazon.com/bedrock/) capability that lets you harness the Foundation Model's (FM's) 7 | reasoning skills to execute multi-steps business tasks using natural language. You can simply state your problem, 8 | like “help me update my product catalogue” and the agent breaks down the problem using the FM’s reasoning capabilities 9 | and executes the steps to fulfill your request. You set up an agent with access to your organization’s enterprise systems, 10 | processes, knowledge bases, and some building block functions. Then the agent comes up with the logic, figures out what APIs 11 | to call and when to call them, and completes the transactions in the right sequence. When an agent needs a piece of information 12 | from the user, it automatically asks the user for those additional details using natural language. And the best part about agents — 13 | it’s leveraging the most up-to-date information you have and gives you relevant answers securely and privately. 14 | 15 | An agent consists of the following components: 16 | 17 | * **Foundation model** – You choose a foundation model that the agent invokes to interpret user input and subsequent prompts in its orchestration process, and to generate responses and follow-up steps in its process. 18 | * **Instructions** – You write up instructions that describe what the agent is designed to do. With advanced prompts, you can further customize instructions for the agent at every step of orchestration and include Lambda functions to parse the output of each step. 19 | * **(Optional) Action groups** – You define the actions that the agent can help the user performing. An action group is composed of actions and the action executor. You can define actions by passing the function details via JSON object or using an API Schema file. You can execute the actions using a Lambda function, or you can return the control to the application and handle the function execution outside of Agents. 20 | * **(Optional) Knowledge bases for Amazon Bedrock** – Associate [Knowledge bases for Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) with an agent to allow it to query the knowledge base for extra context to augment response generation and input into steps of the orchestration process. 21 | * **(Optional) Guardrails for Amazon Bedrock** Associate [Guardrails for Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) with an agent to safeguard your Agent based on use cases and Responsible AI policies 22 | 23 | The following image schematizes the components of your agent. 24 | 25 | 26 | 27 | In build-time, all these components are gathered to construct base prompts for the agent in order to carry out orchestration until the user request is completed. With advanced prompts, you can modify these base prompts with additional logic and few-shot examples to improve accuracy for each step of agent invocation. The base prompt templates contain instructions, action descriptions, knowledge base descriptions, and conversation history, all of which you can customize to modify the agent to the best of your needs. You then prepare your agent, which packages all the components of the agents, including security configurations, and brings the agent into a state where it is ready for testing in runtime. 28 | 29 | 30 | ## Audience 31 | 32 | Architects, data scientists and developer who want to learn how to use Agents for Amazon Bedrock to create generative AI applications. 33 | From the simplest instruction only agent to complex assistants that combine Action Groups with Knowledge Bases, you can use the power of agents to quickly develop your Generative API application. 34 | 35 | ## Workshop Notebooks 36 | During this workshop, you will cover three modules: 37 | 38 | 1. **Building Agents for Bedrock using Boto3 SDK**: In this module, you will create agents for Bedrock programmatically using the restaurant agent example. This module uses the `01_create_agent.ipynb` file 39 | 2. **Integrating Knowledge Bases to your Agents**: In this module, you will create a Knowledge Base and update your restaurant agent to connect this knowledge via Boto3 SDK. This module uses the `02_associate_knowledge_base_to_agent.ipynb` file 40 | 3. **Test your Agent**: In this module, you will invoke your agent with different prompts. This module uses the `03_invoke_agent.ipynb` file 41 | 4. **Clean up Resources**: In the last module you will dissociate all resources from your agent and delete them. This module uses the `04_clean_up_agent_resources.ipynb` file 42 | 43 | ## Setup 44 | Before running any of the labs in this section ensure you've installed the requirements from `requirements.txt` file. 45 | 46 | ## Architecture 47 | In this lab we will create a restaurant assistant agent that connects with a Knowledge Base for Amazon Bedrock containing the restaurant's different menus. 48 | This agent also connects to an action group that provides functionalities for handling the table booking in this restaurant. 49 | ![Agents architect](images/agent-architecture.png) 50 | 51 | The action group created in this example, uses Feature Definition to define the functionalities for `create_booking`, `get_booking_details` and `delete_booking`. 52 | The action group execution connects with a Lambda function that interacts with an Amazon DynamoDB table. 53 | 54 | ## Agents deep dive workshop 55 | The Agents for Amazon Bedrock labs in this workshop provide a high-level overview of the Agents content. 56 | For a deep dive into Agents concepts, including concepts such as return of control, advanced prompts and others, you can check the [Agents for Amazon Bedrock Workshop](https://catalog.workshops.aws/agents-for-amazon-bedrock/en-US) -------------------------------------------------------------------------------- /05_Agents/agent.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | import time 4 | import zipfile 5 | import logging 6 | import pprint 7 | from io import BytesIO 8 | 9 | iam_client = boto3.client('iam') 10 | sts_client = boto3.client('sts') 11 | session = boto3.session.Session() 12 | region = session.region_name 13 | account_id = sts_client.get_caller_identity()["Account"] 14 | dynamodb_client = boto3.client('dynamodb') 15 | dynamodb_resource = boto3.resource('dynamodb') 16 | lambda_client = boto3.client('lambda') 17 | bedrock_agent_client = boto3.client('bedrock-agent') 18 | bedrock_agent_runtime_client = boto3.client('bedrock-agent-runtime') 19 | logging.basicConfig(format='[%(asctime)s] p%(process)s {%(filename)s:%(lineno)d} %(levelname)s - %(message)s', level=logging.INFO) 20 | logger = logging.getLogger(__name__) 21 | 22 | 23 | def create_dynamodb(table_name): 24 | try: 25 | table = dynamodb_resource.create_table( 26 | TableName=table_name, 27 | KeySchema=[ 28 | { 29 | 'AttributeName': 'booking_id', 30 | 'KeyType': 'HASH' 31 | } 32 | ], 33 | AttributeDefinitions=[ 34 | { 35 | 'AttributeName': 'booking_id', 36 | 'AttributeType': 'S' 37 | } 38 | ], 39 | BillingMode='PAY_PER_REQUEST' # Use on-demand capacity mode 40 | ) 41 | 42 | # Wait for the table to be created 43 | print(f'Creating table {table_name}...') 44 | table.wait_until_exists() 45 | print(f'Table {table_name} created successfully!') 46 | except dynamodb_client.exceptions.ResourceInUseException: 47 | print(f'Table {table_name} already exists, skipping table creation step') 48 | 49 | 50 | def create_lambda(lambda_function_name, lambda_iam_role): 51 | # add to function 52 | 53 | # Package up the lambda function code 54 | s = BytesIO() 55 | z = zipfile.ZipFile(s, 'w') 56 | z.write("lambda_function.py") 57 | z.close() 58 | zip_content = s.getvalue() 59 | try: 60 | # Create Lambda Function 61 | lambda_function = lambda_client.create_function( 62 | FunctionName=lambda_function_name, 63 | Runtime='python3.12', 64 | Timeout=60, 65 | Role=lambda_iam_role['Role']['Arn'], 66 | Code={'ZipFile': zip_content}, 67 | Handler='lambda_function.lambda_handler' 68 | ) 69 | except lambda_client.exceptions.ResourceConflictException: 70 | print("Lambda function already exists, retrieving it") 71 | lambda_function = lambda_client.get_function( 72 | FunctionName=lambda_function_name 73 | ) 74 | lambda_function = lambda_function['Configuration'] 75 | 76 | return lambda_function 77 | 78 | 79 | def create_lambda_role(agent_name, dynamodb_table_name): 80 | lambda_function_role = f'{agent_name}-lambda-role' 81 | dynamodb_access_policy_name = f'{agent_name}-dynamodb-policy' 82 | # Create IAM Role for the Lambda function 83 | try: 84 | assume_role_policy_document = { 85 | "Version": "2012-10-17", 86 | "Statement": [ 87 | { 88 | "Effect": "Allow", 89 | "Principal": { 90 | "Service": "lambda.amazonaws.com" 91 | }, 92 | "Action": "sts:AssumeRole" 93 | } 94 | ] 95 | } 96 | 97 | assume_role_policy_document_json = json.dumps(assume_role_policy_document) 98 | 99 | lambda_iam_role = iam_client.create_role( 100 | RoleName=lambda_function_role, 101 | AssumeRolePolicyDocument=assume_role_policy_document_json 102 | ) 103 | 104 | # Pause to make sure role is created 105 | time.sleep(10) 106 | except iam_client.exceptions.EntityAlreadyExistsException: 107 | lambda_iam_role = iam_client.get_role(RoleName=lambda_function_role) 108 | 109 | # Attach the AWSLambdaBasicExecutionRole policy 110 | iam_client.attach_role_policy( 111 | RoleName=lambda_function_role, 112 | PolicyArn='arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' 113 | ) 114 | 115 | # Create a policy to grant access to the DynamoDB table 116 | dynamodb_access_policy = { 117 | "Version": "2012-10-17", 118 | "Statement": [ 119 | { 120 | "Effect": "Allow", 121 | "Action": [ 122 | "dynamodb:GetItem", 123 | "dynamodb:PutItem", 124 | "dynamodb:DeleteItem" 125 | ], 126 | "Resource": "arn:aws:dynamodb:{}:{}:table/{}".format( 127 | region, account_id, dynamodb_table_name 128 | ) 129 | } 130 | ] 131 | } 132 | 133 | # Create the policy 134 | dynamodb_access_policy_json = json.dumps(dynamodb_access_policy) 135 | try: 136 | dynamodb_access_policy = iam_client.create_policy( 137 | PolicyName=dynamodb_access_policy_name, 138 | PolicyDocument=dynamodb_access_policy_json 139 | ) 140 | except iam_client.exceptions.EntityAlreadyExistsException: 141 | dynamodb_access_policy = iam_client.get_policy( 142 | PolicyArn=f"arn:aws:iam::{account_id}:policy/{dynamodb_access_policy_name}" 143 | ) 144 | 145 | # Attach the policy to the Lambda function's role 146 | iam_client.attach_role_policy( 147 | RoleName=lambda_function_role, 148 | PolicyArn=dynamodb_access_policy['Policy']['Arn'] 149 | ) 150 | return lambda_iam_role 151 | 152 | 153 | def invoke_agent_helper(query, session_id, agent_id, alias_id, enable_trace=False, session_state=None): 154 | end_session: bool = False 155 | if not session_state: 156 | session_state = {} 157 | 158 | # invoke the agent API 159 | agent_response = bedrock_agent_runtime_client.invoke_agent( 160 | inputText=query, 161 | agentId=agent_id, 162 | agentAliasId=alias_id, 163 | sessionId=session_id, 164 | enableTrace=enable_trace, 165 | endSession=end_session, 166 | sessionState=session_state 167 | ) 168 | 169 | if enable_trace: 170 | logger.info(pprint.pprint(agent_response)) 171 | 172 | event_stream = agent_response['completion'] 173 | try: 174 | for event in event_stream: 175 | if 'chunk' in event: 176 | data = event['chunk']['bytes'] 177 | if enable_trace: 178 | logger.info(f"Final answer ->\n{data.decode('utf8')}") 179 | agent_answer = data.decode('utf8') 180 | return agent_answer 181 | # End event indicates that the request finished successfully 182 | elif 'trace' in event: 183 | if enable_trace: 184 | logger.info(json.dumps(event['trace'], indent=2, default=str)) 185 | else: 186 | raise Exception("unexpected event.", event) 187 | except Exception as e: 188 | raise Exception("unexpected event.", e) 189 | 190 | 191 | def create_agent_role(agent_name, agent_foundation_model, kb_id=None): 192 | agent_bedrock_allow_policy_name = f"{agent_name}-ba" 193 | agent_role_name = f'AmazonBedrockExecutionRoleForAgents_{agent_name}' 194 | # Create IAM policies for agent 195 | statements = [ 196 | { 197 | "Sid": "AmazonBedrockAgentBedrockFoundationModelPolicy", 198 | "Effect": "Allow", 199 | "Action": "bedrock:InvokeModel", 200 | "Resource": [ 201 | f"arn:aws:bedrock:{region}::foundation-model/{agent_foundation_model}" 202 | ] 203 | } 204 | ] 205 | # add Knowledge Base retrieve and retrieve and generate permissions if agent has KB attached to it 206 | if kb_id: 207 | statements.append( 208 | { 209 | "Sid": "QueryKB", 210 | "Effect": "Allow", 211 | "Action": [ 212 | "bedrock:Retrieve", 213 | "bedrock:RetrieveAndGenerate" 214 | ], 215 | "Resource": [ 216 | f"arn:aws:bedrock:{region}:{account_id}:knowledge-base/{kb_id}" 217 | ] 218 | } 219 | ) 220 | 221 | bedrock_agent_bedrock_allow_policy_statement = { 222 | "Version": "2012-10-17", 223 | "Statement": statements 224 | } 225 | 226 | bedrock_policy_json = json.dumps(bedrock_agent_bedrock_allow_policy_statement) 227 | try: 228 | agent_bedrock_policy = iam_client.create_policy( 229 | PolicyName=agent_bedrock_allow_policy_name, 230 | PolicyDocument=bedrock_policy_json 231 | ) 232 | except iam_client.exceptions.EntityAlreadyExistsException: 233 | agent_bedrock_policy = iam_client.get_policy( 234 | PolicyArn=f"arn:aws:iam::{account_id}:policy/{agent_bedrock_allow_policy_name}" 235 | ) 236 | 237 | # Create IAM Role for the agent and attach IAM policies 238 | assume_role_policy_document = { 239 | "Version": "2012-10-17", 240 | "Statement": [{ 241 | "Effect": "Allow", 242 | "Principal": { 243 | "Service": "bedrock.amazonaws.com" 244 | }, 245 | "Action": "sts:AssumeRole" 246 | }] 247 | } 248 | 249 | assume_role_policy_document_json = json.dumps(assume_role_policy_document) 250 | try: 251 | agent_role = iam_client.create_role( 252 | RoleName=agent_role_name, 253 | AssumeRolePolicyDocument=assume_role_policy_document_json 254 | ) 255 | 256 | # Pause to make sure role is created 257 | time.sleep(10) 258 | except iam_client.exceptions.EntityAlreadyExistsException: 259 | agent_role = iam_client.get_role( 260 | RoleName=agent_role_name, 261 | ) 262 | 263 | iam_client.attach_role_policy( 264 | RoleName=agent_role_name, 265 | PolicyArn=agent_bedrock_policy['Policy']['Arn'] 266 | ) 267 | return agent_role 268 | 269 | 270 | def delete_agent_roles_and_policies(agent_name, kb_policy_name): 271 | agent_bedrock_allow_policy_name = f"{agent_name}-ba" 272 | agent_role_name = f'AmazonBedrockExecutionRoleForAgents_{agent_name}' 273 | dynamodb_access_policy_name = f'{agent_name}-dynamodb-policy' 274 | lambda_function_role = f'{agent_name}-lambda-role' 275 | 276 | for policy in [agent_bedrock_allow_policy_name, kb_policy_name]: 277 | try: 278 | iam_client.detach_role_policy( 279 | RoleName=agent_role_name, 280 | PolicyArn=f'arn:aws:iam::{account_id}:policy/{policy}' 281 | ) 282 | except Exception as e: 283 | print(f"Could not detach {policy} from {agent_role_name}") 284 | print(e) 285 | 286 | for policy in [dynamodb_access_policy_name]: 287 | try: 288 | iam_client.detach_role_policy( 289 | RoleName=lambda_function_role, 290 | PolicyArn=f'arn:aws:iam::{account_id}:policy/{policy}' 291 | ) 292 | except Exception as e: 293 | print(f"Could not detach {policy} from {lambda_function_role}") 294 | print(e) 295 | 296 | try: 297 | iam_client.detach_role_policy( 298 | RoleName=lambda_function_role, 299 | PolicyArn='arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' 300 | ) 301 | except Exception as e: 302 | print(f"Could not detach AWSLambdaBasicExecutionRole from {lambda_function_role}") 303 | print(e) 304 | 305 | for role_name in [agent_role_name, lambda_function_role]: 306 | try: 307 | iam_client.delete_role( 308 | RoleName=role_name 309 | ) 310 | except Exception as e: 311 | print(f"Could not delete role {role_name}") 312 | print(e) 313 | 314 | for policy in [agent_bedrock_allow_policy_name, kb_policy_name, dynamodb_access_policy_name]: 315 | try: 316 | iam_client.delete_policy( 317 | PolicyArn=f'arn:aws:iam::{account_id}:policy/{policy}' 318 | ) 319 | except Exception as e: 320 | print(f"Could not delete policy {policy}") 321 | print(e) 322 | 323 | 324 | def clean_up_resources( 325 | table_name, lambda_function, lambda_function_name, agent_action_group_response, agent_functions, 326 | agent_id, kb_id, alias_id 327 | ): 328 | action_group_id = agent_action_group_response['agentActionGroup']['actionGroupId'] 329 | action_group_name = agent_action_group_response['agentActionGroup']['actionGroupName'] 330 | # Delete Agent Action Group, Agent Alias, and Agent 331 | try: 332 | bedrock_agent_client.update_agent_action_group( 333 | agentId=agent_id, 334 | agentVersion='DRAFT', 335 | actionGroupId= action_group_id, 336 | actionGroupName=action_group_name, 337 | actionGroupExecutor={ 338 | 'lambda': lambda_function['FunctionArn'] 339 | }, 340 | functionSchema={ 341 | 'functions': agent_functions 342 | }, 343 | actionGroupState='DISABLED', 344 | ) 345 | bedrock_agent_client.disassociate_agent_knowledge_base( 346 | agentId=agent_id, 347 | agentVersion='DRAFT', 348 | knowledgeBaseId=kb_id 349 | ) 350 | bedrock_agent_client.delete_agent_action_group( 351 | agentId=agent_id, 352 | agentVersion='DRAFT', 353 | actionGroupId=action_group_id 354 | ) 355 | bedrock_agent_client.delete_agent_alias( 356 | agentAliasId=alias_id, 357 | agentId=agent_id 358 | ) 359 | bedrock_agent_client.delete_agent(agentId=agent_id) 360 | print(f"Agent {agent_id}, Agent Alias {alias_id}, and Action Group have been deleted.") 361 | except Exception as e: 362 | print(f"Error deleting Agent resources: {e}") 363 | 364 | # Delete Lambda function 365 | try: 366 | lambda_client.delete_function(FunctionName=lambda_function_name) 367 | print(f"Lambda function {lambda_function_name} has been deleted.") 368 | except Exception as e: 369 | print(f"Error deleting Lambda function {lambda_function_name}: {e}") 370 | 371 | # Delete DynamoDB table 372 | try: 373 | dynamodb_client.delete_table(TableName=table_name) 374 | print(f"Table {table_name} is being deleted...") 375 | waiter = dynamodb_client.get_waiter('table_not_exists') 376 | waiter.wait(TableName=table_name) 377 | print(f"Table {table_name} has been deleted.") 378 | except Exception as e: 379 | print(f"Error deleting table {table_name}: {e}") 380 | -------------------------------------------------------------------------------- /05_Agents/images/agent-architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/05_Agents/images/agent-architecture.png -------------------------------------------------------------------------------- /05_Agents/images/agent-architecture_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/05_Agents/images/agent-architecture_1.png -------------------------------------------------------------------------------- /05_Agents/images/agent-orchestration-flow-pink.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/05_Agents/images/agent-orchestration-flow-pink.png -------------------------------------------------------------------------------- /05_Agents/images/agent_components.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/05_Agents/images/agent_components.png -------------------------------------------------------------------------------- /05_Agents/kb_documents/Restaurant_Childrens_Menu.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/05_Agents/kb_documents/Restaurant_Childrens_Menu.pdf -------------------------------------------------------------------------------- /05_Agents/kb_documents/Restaurant_Dinner_Menu.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/05_Agents/kb_documents/Restaurant_Dinner_Menu.pdf -------------------------------------------------------------------------------- /05_Agents/kb_documents/Restaurant_week_specials.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/05_Agents/kb_documents/Restaurant_week_specials.pdf -------------------------------------------------------------------------------- /05_Agents/requirements.txt: -------------------------------------------------------------------------------- 1 | boto3 2 | opensearch-py 3 | botocore 4 | awscli 5 | retrying -------------------------------------------------------------------------------- /06_OpenSource_examples/README.md: -------------------------------------------------------------------------------- 1 | # Open-Source Module: Amazon Bedrock Integration with LangChain and Agentic Frameworks 2 | 3 | Welcome to the **Open-Source Module**, where we demonstrate how to use [Amazon Bedrock](https://aws.amazon.com/bedrock/) with popular open-source libraries and frameworks. This repository includes a series of Jupyter notebooks that range from **high-level LangChain use cases** to **advanced agentic frameworks** with retrieval-augmented generation (RAG) and agent evaluation. Additionally, we provide an example using **CrewAI** for building a React-style agents, contrasting with the Agentic workflow approach in **LangGraph**. 4 | 5 | ## Table of Contents 6 | - [Overview](#overview) 7 | - [Prerequisites](#prerequisites) 8 | - [Installation](#installation) 9 | - [High-Level Use Cases Notebooks](#high-level-use-cases) 10 | - [Agentic Frameworks, Evaluations and RAG](#agentic-frameworks-evaluations-and-rag) 11 | 12 | --- 13 | 14 | ## Overview 15 | 16 | **Amazon Bedrock** is a fully managed service that lets you easily build and scale generative AI applications with foundation models (FMs) from top AI providers, all accessible via a single API. In this module, we: 17 | 18 | Illustrate **high-level examples** of text generation, code translation, and summarization using Bedrock with **LangChain**. 19 | Dive into **advanced agent-based setups** using RAG (retrieval-augmented generation), multi-agent frameworks, agent evaluation, and React-based agents (via **CrewAI**) as an alternative to **LangGraph**’s workflow approach. 20 | 21 | --- 22 | 23 | ## Prerequisites 24 | 25 | **AWS Account & Credentials** 26 | You’ll need access to Amazon Bedrock. Ensure your AWS CLI or environment variables are properly configured to allow these notebooks to authenticate. 27 | 28 | **Python 3.12+** 29 | Install and use a version of Python that is 3.12 or higher. 30 | 31 | **JupyterLab or Jupyter Notebook** 32 | You can run the `.ipynb` notebooks locally or in a managed environment (e.g., Amazon SageMaker AI). 33 | 34 | --- 35 | 36 | ## Installation 37 | 38 | **Clone the Repository** 39 | 40 | ```bash 41 | git clone https://github.com/aws-samples/amazon-bedrock-workshop.git 42 | cd amazon-bedrock-workshop/06_OpenSource_examples 43 | pip install -r requirements.txt 44 | ``` 45 | --- 46 | ## Notebook Descriptions 47 | 48 | These notebooks are each independent and can be executed in any order. 49 | 50 | ### High-Level Use Cases 51 | 52 | These notebooks are located in the `text-generation-with-langchain/` directory and demonstrate core ways to use Amazon Bedrock with LangChain: 53 | - ***00_simple_chatbot.ipynb***: 54 | Create a multi-lingal greeter chatbot using langchain with conversation retention and prompt templating. 55 | - ***01_zero_shot_generation.ipynb***: 56 | Zero-shot text generation with foundation models on Bedrock, wrapped in LangChain’s prompt interface. 57 | - ***02_code_interpret_w_langchain.ipynb***: 58 | Code interpretation and generation, showing how to integrate Bedrock models with LangChain abstractions for developer workflows. 59 | - ***03_code_translate_w_langchain.ipynb***: 60 | Demonstrates code translation from one programming language to another, powered by Bedrock + LangChain’s flexible prompt layering. 61 | - ***04_long_text_summarization_using_LCEL_chains_on_Langchain.ipynb***: 62 | Explores how to summarize lengthy documents or text using LCEL (Language Chain Execution Logic) in LangChain, backed by Bedrock models. 63 | 64 | ### Agentic Frameworks, Evaluations and RAG 65 | 66 | Beyond the high-level examples, you’ll find a set of notebooks that delve into agent frameworks, retrieval-augmented generation, and agent evaluation (using **Ragas**). Some of these also highlight the difference between a React agent approach (using **CrewAI**) and a workflow agentic approach (using **LangGraph**). 67 | - ***advance-langragph-multi-agent-setup.ipynb***: 68 | Shows how to set up a multi-agent environment leveraging LangGraph workflows and Amazon Bedrock. 69 | - ***find-relevant-information-using-RAG.ipynb***: 70 | Demonstrates retrieval-augmented generation (RAG) with Bedrock for more accurate and context-rich responses. 71 | - ***intermediate-langragph-agent-setup-w-tools.ipynb***: 72 | Focuses on an intermediate agentic workflow in LangGraph, integrating additional tools to handle complex tasks. 73 | - ***ragas-agent-evaluation.ipynb***: 74 | Explains evaluating agent performance using Ragas, focusing on retrieval and generation quality with Bedrock. 75 | - ***simple-crewai-agent-setup.ipynb***: 76 | Illustrates a minimal React agent approach using CrewAI, contrasting with LangGraph workflows. 77 | - ***simple-langragph-agent-setup.ipynb***: 78 | Provides a starting point for building a single-agent flow with LangGraph, leveraging Bedrock behind the scenes. 79 | --- 80 | AWS Configuration 81 | Make sure your environment is set up with AWS credentials that can call Amazon Bedrock. If necessary, export environment variables or configure the AWS CLI. 82 | Run & Explore 83 | Execute the cells in each notebook sequentially. You can modify prompts, model parameters, agent setups, or workflows to see how results change. 84 | 85 | For more information on the fine-grained action and resource permissions in Bedrock, check out the [Bedrock Developer Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html). 86 | 87 | -------------------------------------------------------------------------------- /06_OpenSource_examples/data/section_doc_store.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/section_doc_store.pkl -------------------------------------------------------------------------------- /06_OpenSource_examples/data/section_vector_store.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/section_vector_store.pkl -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/asheville_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/asheville_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/atlanta_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/atlanta_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/austin_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/austin_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/baltimore_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/baltimore_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/boston_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/boston_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/charleston_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/charleston_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/charlotte_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/charlotte_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/chicago_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/chicago_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/cincinnati_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/cincinnati_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/dallas_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/dallas_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/denver_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/denver_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/jacksonville_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/jacksonville_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/data/travel_guides/kansas_city_travel_guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/data/travel_guides/kansas_city_travel_guide.pdf -------------------------------------------------------------------------------- /06_OpenSource_examples/images/Hotel_booking_confirmation_light.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/Hotel_booking_confirmation_light.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/RAG-travellers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/RAG-travellers.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/ReACT_agents.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/ReACT_agents.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/ReWoO_agents.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/ReWoO_agents.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/agent_action_thought.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/agent_action_thought.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/agent_alignment_chart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/agent_alignment_chart.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/agents_itinerary.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/agents_itinerary.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/agents_memory_light.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/agents_memory_light.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/amsterdam.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/amsterdam.jpeg -------------------------------------------------------------------------------- /06_OpenSource_examples/images/chatbot_bedrock.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/chatbot_bedrock.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/converse_api.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/converse_api.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/graph_state_light.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/graph_state_light.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/human-in-the-loop.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/human-in-the-loop.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/multi-agent-travel-bot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/multi-agent-travel-bot.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/multi_memory_light.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/multi_memory_light.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/rag-architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/rag-architecture.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/rag-workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/rag-workflow.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/react_tool_call.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/react_tool_call.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/short-vs-long.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/short-vs-long.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/travel_agent_light.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/travel_agent_light.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/vegas.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/vegas.png -------------------------------------------------------------------------------- /06_OpenSource_examples/images/weather.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/images/weather.jpg -------------------------------------------------------------------------------- /06_OpenSource_examples/requirements.txt: -------------------------------------------------------------------------------- 1 | langchain 2 | langchain_aws 3 | langchain_core 4 | langchain 5 | ragas 6 | transformers==4.49.0 7 | pandas 8 | langgraph 9 | langchain_community 10 | faker==36.1.1 11 | faiss-cpu==1.10.0 12 | crewai==0.102.0 13 | PyPDF2==3.0.1 14 | duckduckgo-search==7.5.0 15 | wikipedia 16 | rapidfuzz==3.12.2 -------------------------------------------------------------------------------- /06_OpenSource_examples/text-generation-with-langchain/00_simple_chatbot.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "bc96dc43-8bbf-4d82-9010-34b802ae4798", 6 | "metadata": {}, 7 | "source": [ 8 | "# Coversational Interface - Chatbot with Claude LLM\n", 9 | "In this notebook, we will build a chatbot using the Foundation Models (FMs) in Amazon Bedrock. For our use-case we use Amazon Nova Lite as our FM for building the chatbot and InMemoryChatMessageHistory to store the conversation history.\n", 10 | "\n", 11 | "## Overview\n", 12 | "\n", 13 | "Conversational interfaces such as chatbots and virtual assistants can be used to enhance the user experience for your customers. Chatbots uses natural language processing (NLP) and machine learning algorithms to understand and respond to user queries. Chatbots can be used in a variety of applications, such as customer service, sales, and e-commerce, to provide quick and efficient responses to users. They can be accessed through various channels such as websites, social media platforms, and messaging apps.\n", 14 | "\n", 15 | "![Using Amazon Bedrock to support a multi-turn conversation with AI](.././images/chatbot_bedrock.png)\n" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": null, 21 | "id": "3ac42792-dbd3-40a1-ab0d-50cdaca25fbe", 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "!pip install --upgrade -q -r requirements.txt" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "id": "c9437c62-8c7f-41b4-8b3e-8af03d70441a", 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "# Restart kernel\n", 36 | "from IPython.core.display import HTML\n", 37 | "HTML(\"\")" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "id": "ac9e2e43-c77c-4c8e-a581-f43bfd89fe86", 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "import boto3\n", 48 | "import botocore" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "id": "e892226c-1117-461a-b72a-a5616c7a45ea", 55 | "metadata": {}, 56 | "outputs": [], 57 | "source": [ 58 | "from langchain_aws.chat_models import ChatBedrock\n", 59 | "from langchain_core.runnables.history import RunnableWithMessageHistory\n", 60 | "from langchain_core.chat_history import InMemoryChatMessageHistory" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "id": "80151c5c-d919-46d2-83d3-c52d0c550bf5", 66 | "metadata": {}, 67 | "source": [ 68 | "### Set up" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": null, 74 | "id": "b0761cfa-a2c2-4d52-be5a-dddae776a9f9", 75 | "metadata": {}, 76 | "outputs": [], 77 | "source": [ 78 | "boto3_session = boto3.session.Session()\n", 79 | "region = boto3_session.region_name\n", 80 | "\n", 81 | "# the statement below can be used to override the region in the session\n", 82 | "#region = \"us-west-2\"" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "id": "fc1b7041-c816-4764-8672-4b7057e7e705", 88 | "metadata": {}, 89 | "source": [ 90 | "### Run the chatbot" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "id": "32bc34ff-0ee1-4fd2-abff-d00edc0586b0", 96 | "metadata": {}, 97 | "source": [ 98 | "Set up parameters for the model and create a client" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": null, 104 | "id": "8fb59de4-fcdf-4cfd-b243-2f16da715516", 105 | "metadata": {}, 106 | "outputs": [], 107 | "source": [ 108 | "model = \"us.amazon.nova-lite-v1:0\"\n", 109 | "temperature = 0.1" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "id": "28852f4f-d7e1-49a8-959b-1a8eb8e8083d", 116 | "metadata": {}, 117 | "outputs": [], 118 | "source": [ 119 | "llm_chat = ChatBedrock(\n", 120 | " model_id=model, \n", 121 | " model_kwargs={\"temperature\": temperature},\n", 122 | " region_name=region\n", 123 | ")\n" 124 | ] 125 | }, 126 | { 127 | "cell_type": "markdown", 128 | "id": "e94cdcab-ce24-4fb0-ba5f-cd4359690fdc", 129 | "metadata": {}, 130 | "source": [ 131 | "Passing conversation state into and out a chain is vital when building a chatbot. The [RunnableWithMessageHistory class](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html) lets us add message history to certain types of chains. It wraps another Runnable and manages the chat message history for it. Specifically, it loads previous messages in the conversation BEFORE passing it to the Runnable, and it saves the generated response as a message AFTER calling the runnable. This class also enables multiple conversations by saving each conversation with a session_id - it then expects a session_id to be passed in the config when calling the runnable, and uses that to look up the relevant conversation history. You have to implement a function like the get_session_history() which will take as input the session-id and return the conversation for the session. Here we have a simple implementation of the conversation history, using the InMemoryChatMessageHistory class" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": null, 137 | "id": "18901bf2-b71f-4941-9a39-f53d05d0ae38", 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [ 141 | "store = {}\n", 142 | "\n", 143 | "def get_session_history(session_id):\n", 144 | " if session_id not in store:\n", 145 | " store[session_id] = InMemoryChatMessageHistory()\n", 146 | " return store[session_id]\n", 147 | "\n", 148 | "with_message_history = RunnableWithMessageHistory(llm_chat, \n", 149 | " get_session_history)\n" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": null, 155 | "id": "f4bb8438-b0b0-42a3-aacd-53d1fa316d8d", 156 | "metadata": {}, 157 | "outputs": [], 158 | "source": [ 159 | "from langchain_core.messages import HumanMessage\n", 160 | "\n", 161 | "response = with_message_history.invoke(\n", 162 | " [HumanMessage(content=\"hi - i am bob!\")],\n", 163 | " config={\"configurable\": {\"session_id\": \"1\"}},\n", 164 | ")\n", 165 | "response.content" 166 | ] 167 | }, 168 | { 169 | "cell_type": "code", 170 | "execution_count": null, 171 | "id": "60275cab-a8d8-470a-bfa3-8b78636d28d7", 172 | "metadata": {}, 173 | "outputs": [], 174 | "source": [ 175 | "response = with_message_history.invoke(\n", 176 | " [HumanMessage(content=\"whats my name?\")],\n", 177 | " config={\"configurable\": {\"session_id\": \"1\"}},\n", 178 | ")\n", 179 | "response.content" 180 | ] 181 | }, 182 | { 183 | "attachments": {}, 184 | "cell_type": "markdown", 185 | "id": "4c975f46-0e2a-4823-bdd9-be2a8ec9ce9d", 186 | "metadata": {}, 187 | "source": [ 188 | "At this point the store has 1 key (session_id = '1') and its value is a list with 4 messages:\n", 189 | " [HumanMessage, AIMessage, HumanMessage, AIMessage]\n", 190 | "\n", 191 | "All langchain messages have 3 properties: a role, content, response_metadata. The HumanMessage returns 1 property, the *content* (e.g. the message that the user passed) whereas the AIMessage also returns non-empty *response_metadata*. It also includes the property *usage_metadata*, a dictionary with these keys: input_tokens, output_tokens, total_tokens (these are also included in the response_metadata) \n" 192 | ] 193 | }, 194 | { 195 | "cell_type": "code", 196 | "execution_count": null, 197 | "id": "8f45c386-dca0-4b9d-98fd-4df46e26b52e", 198 | "metadata": {}, 199 | "outputs": [], 200 | "source": [ 201 | "print(store)\n", 202 | "# uncomment the following line to take a closer look at the message associated with session_id='1'\n", 203 | "#store['1'].messages[1]" 204 | ] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "id": "6bc8ebfd-92bc-4ac7-b537-9510a8b0f2bc", 209 | "metadata": {}, 210 | "source": [ 211 | "### Create a Multi-Lingual Greeter Chatbot! \n", 212 | "Building on our above example the RunnableWithMessageHistory class from langchain which serves to:\n", 213 | "1) retain `InMemoryChatMessageHistory` for each follow up message within the same session ID\n", 214 | "2) call the LLM ([*Runnable*](https://python.langchain.com/v0.1/docs/expression_language/interface/))\n", 215 | "3) write the AI response message back into the `InMemoryChatMessageHistory`\n", 216 | "\n", 217 | "Now we will introduce [ChatPromptTemplate](https://python.langchain.com/api_reference/core/prompts/langchain_core.prompts.chat.ChatPromptTemplate.html) to configure our chat application to be multi-lingual!" 218 | ] 219 | }, 220 | { 221 | "cell_type": "code", 222 | "execution_count": null, 223 | "id": "e5151501-1ec6-4400-9cf8-82194d0836e4", 224 | "metadata": {}, 225 | "outputs": [], 226 | "source": [ 227 | "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n", 228 | "\n", 229 | "prompt = ChatPromptTemplate.from_messages([\n", 230 | " (\"system\",\"You're an assistant who speaks in {language}. Translate the user input\"),\n", 231 | " MessagesPlaceholder(variable_name=\"history\"),\n", 232 | " (\"human\", \"{question}\"),\n", 233 | "])\n", 234 | "\n", 235 | "chain = prompt | llm_chat\n", 236 | "\n", 237 | "chain_with_history = RunnableWithMessageHistory(\n", 238 | " chain,\n", 239 | " get_session_history,\n", 240 | " input_messages_key=\"question\",\n", 241 | " history_messages_key=\"history\",\n", 242 | ")\n", 243 | "\n", 244 | "lang = \"French\"\n", 245 | "print(chain_with_history.invoke(\n", 246 | " {\"language\": lang, \"question\": \"Hi my name is John\"},\n", 247 | " config={\"configurable\": {\"session_id\": \"2\"}}\n", 248 | "))" 249 | ] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": null, 254 | "id": "5c2288d1-4dc8-4de2-8e3a-a421bf40a1d5", 255 | "metadata": {}, 256 | "outputs": [], 257 | "source": [ 258 | "print(chain_with_history.invoke(\n", 259 | " {\"language\": lang, \"question\": \"What is my name?\"},\n", 260 | " config={\"configurable\": {\"session_id\": \"2\"}}\n", 261 | "))" 262 | ] 263 | }, 264 | { 265 | "cell_type": "code", 266 | "execution_count": null, 267 | "id": "21f8bb4e-119e-4aea-a541-260321cabc8b", 268 | "metadata": {}, 269 | "outputs": [], 270 | "source": [ 271 | "print(store['2'].messages)" 272 | ] 273 | } 274 | ], 275 | "metadata": { 276 | "kernelspec": { 277 | "display_name": "Python 3 (ipykernel)", 278 | "language": "python", 279 | "name": "python3" 280 | }, 281 | "language_info": { 282 | "codemirror_mode": { 283 | "name": "ipython", 284 | "version": 3 285 | }, 286 | "file_extension": ".py", 287 | "mimetype": "text/x-python", 288 | "name": "python", 289 | "nbconvert_exporter": "python", 290 | "pygments_lexer": "ipython3", 291 | "version": "3.11.11" 292 | } 293 | }, 294 | "nbformat": 4, 295 | "nbformat_minor": 5 296 | } 297 | -------------------------------------------------------------------------------- /06_OpenSource_examples/text-generation-with-langchain/images/bedrock_langchain.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/text-generation-with-langchain/images/bedrock_langchain.jpg -------------------------------------------------------------------------------- /06_OpenSource_examples/text-generation-with-langchain/images/code-interpret-langchain.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/text-generation-with-langchain/images/code-interpret-langchain.png -------------------------------------------------------------------------------- /06_OpenSource_examples/text-generation-with-langchain/images/code-translation-langchain.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/06_OpenSource_examples/text-generation-with-langchain/images/code-translation-langchain.png -------------------------------------------------------------------------------- /06_OpenSource_examples/text-generation-with-langchain/requirements.txt: -------------------------------------------------------------------------------- 1 | langchain 2 | langchain_aws 3 | langchain_core 4 | langchain 5 | ragas 6 | transformers==4.49.0 7 | pandas 8 | langgraph 9 | langchain_community 10 | faker==36.1.1 11 | faiss-cpu==1.10.0 12 | crewai==0.102.0 13 | PyPDF2==3.0.1 14 | duckduckgo-search==7.5.0 15 | wikipedia 16 | rapidfuzz==3.12.2 -------------------------------------------------------------------------------- /06_OpenSource_examples/utils.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import boto3 3 | import pickle 4 | import random 5 | from datetime import datetime, timedelta 6 | from faker import Faker 7 | import sqlite3 8 | import math 9 | import os 10 | 11 | 12 | from collections import Counter 13 | from langchain_core.tools import tool 14 | from langchain_core.runnables.config import RunnableConfig 15 | from langchain_aws import ChatBedrockConverse 16 | from langchain_text_splitters import RecursiveCharacterTextSplitter 17 | from langchain_aws.embeddings.bedrock import BedrockEmbeddings 18 | from langgraph.prebuilt import create_react_agent 19 | from langgraph.checkpoint.memory import MemorySaver 20 | from langchain.tools.retriever import create_retriever_tool 21 | from langchain_community.vectorstores import FAISS 22 | from langchain.retrievers import ParentDocumentRetriever 23 | 24 | 25 | from ragas.messages import HumanMessage as RGHumanMessage 26 | from ragas.messages import AIMessage as RGAIMessage 27 | from ragas.messages import ToolMessage as RGToolMessage 28 | from ragas.messages import ToolCall as RGToolCall 29 | 30 | from io import BytesIO 31 | 32 | def convert_message_langchain_to_ragas(lc_message): 33 | message_dict = lc_message.model_dump() 34 | if message_dict['type'] == 'human': 35 | rg_message = RGHumanMessage(content=message_dict['content']) 36 | if message_dict['type'] == 'ai': 37 | if type(message_dict['content']) == list: 38 | text = list(filter((lambda x: x['type'] == 'text'), message_dict['content'])) 39 | tool = list(filter((lambda x: x['type'] == 'tool_use'), message_dict['content'])) 40 | if len(text) > 0 and len(tool) > 0: 41 | 42 | if len(list(tool[0]['input'].keys())) > 0: 43 | dyn_args = {'query': tool[0]['input'][list(tool[0]['input'].keys())[0]]} 44 | else: 45 | dyn_args = {} 46 | 47 | rg_message = RGAIMessage(content=text[0]['text'], tool_calls=[RGToolCall(name=tool[0]['name'], args= dyn_args)]) 48 | elif len(text) > 0: 49 | rg_message = RGAIMessage(content=text[0]['text']) 50 | elif len(tool) > 0: 51 | rg_message = RGAIMessage(content='', tool_calls=[RGToolCall(name=tool[0]['name'], args={#'id': tool[0]['id'], 52 | 'query': tool[0]['input'][list(tool[0]['input'].keys())[0]]})]) 53 | else: 54 | rg_message = RGAIMessage(content= message_dict['content'], tool_calls=message_dict['tool_calls'], metadata=message_dict['usage_metadata']) 55 | if message_dict['type'] == 'tool': 56 | rg_message = RGToolMessage(content=message_dict['content'], metadata={"tool_name": message_dict['name'], "tool_call_id": message_dict['tool_call_id']}) 57 | return rg_message 58 | 59 | 60 | def create_agent(enable_memory = False): 61 | # ---- ⚠️ Update region for your AWS setup ⚠️ ---- 62 | bedrock_client = boto3.client("bedrock-runtime", region_name="us-west-2") 63 | 64 | 65 | 66 | llm = ChatBedrockConverse( 67 | model="anthropic.claude-3-haiku-20240307-v1:0", 68 | temperature=0, 69 | max_tokens=None, 70 | client=bedrock_client, 71 | # other params... 72 | ) 73 | 74 | def read_travel_data(file_path: str = "data/synthetic_travel_data.csv") -> pd.DataFrame: 75 | """Read travel data from CSV file""" 76 | try: 77 | df = pd.read_csv(file_path) 78 | return df 79 | except FileNotFoundError: 80 | return pd.DataFrame( 81 | columns=["Id", "Name","Current_Location","Age","Past_Travel_Destinations", "Number_of_Trips", "Flight_Number", "Departure_City","Arrival_City","Flight_Date",] 82 | ) 83 | 84 | 85 | @tool 86 | def compare_and_recommend_destination(config: RunnableConfig) -> str: 87 | """This tool is used to check which destinations user has already traveled. 88 | If user has already been to a city then do not recommend that city. 89 | 90 | Returns: 91 | str: Destination to be recommended. 92 | 93 | """ 94 | 95 | df = read_travel_data() 96 | user_id = config.get("configurable", {}).get("user_id") 97 | 98 | if user_id not in df["Id"].values: 99 | return "User not found in the travel database." 100 | 101 | user_data = df[df["Id"] == user_id].iloc[0] 102 | current_location = user_data["Current_Location"] 103 | age = user_data["Age"] 104 | past_destinations = user_data["Past_Travel_Destinations"].split(", ") 105 | 106 | # Get all past destinations of users with similar age (±5 years) and same current location 107 | similar_users = df[(df["Current_Location"] == current_location) & (df["Age"].between(age - 5, age + 5))] 108 | all_destinations = [dest for user_dests in similar_users["Past_Travel_Destinations"].str.split(", ") for dest in user_dests ] 109 | 110 | # Count occurrences of each destination 111 | destination_counts = Counter(all_destinations) 112 | 113 | # Remove user's current location and past destinations from recommendations 114 | for dest in [current_location] + past_destinations: 115 | if dest in destination_counts: 116 | del destination_counts[dest] 117 | 118 | if not destination_counts: 119 | return f"No new recommendations found for users in {current_location} with similar age." 120 | 121 | # Get the most common destination 122 | recommended_destination = destination_counts.most_common(1)[0][0] 123 | 124 | return f"Based on your current location ({current_location}), age ({age}), and past travel data, we recommend visiting {recommended_destination}." 125 | 126 | 127 | embeddings_model = BedrockEmbeddings( 128 | client=bedrock_client, model_id="amazon.titan-embed-text-v1" 129 | ) 130 | 131 | child_splitter = RecursiveCharacterTextSplitter( 132 | separators=["\n", "\n\n"], chunk_size=2000, chunk_overlap=250 133 | ) 134 | 135 | in_memory_store_file = "data/section_doc_store.pkl" 136 | vector_store_file = "data/section_vector_store.pkl" 137 | 138 | store = pickle.load(open(in_memory_store_file, "rb")) 139 | vector_db_buff = BytesIO(pickle.load(open(vector_store_file, "rb"))) 140 | vector_db = FAISS.deserialize_from_bytes( 141 | serialized=vector_db_buff.read(), 142 | embeddings=embeddings_model, 143 | allow_dangerous_deserialization=True, 144 | ) 145 | 146 | retriever = ParentDocumentRetriever( 147 | vectorstore=vector_db, 148 | docstore=store, 149 | child_splitter=child_splitter, 150 | ) 151 | 152 | retriever_tool = create_retriever_tool( 153 | retriever, 154 | "travel_guide", 155 | """Holds information from travel guide books containing city details to find information matching the user's interests in various cities. Only search based on the keyword mentioned in user input. 156 | 157 | Args: 158 | query (str): place to query travel guide. 159 | Returns: 160 | str: Information about destination from travel guide. 161 | 162 | """, 163 | ) 164 | 165 | tools = [compare_and_recommend_destination, retriever_tool] 166 | 167 | if enable_memory: 168 | 169 | memory = MemorySaver() 170 | agent = create_react_agent(llm, tools, checkpointer = memory) 171 | 172 | else: 173 | agent = create_react_agent(llm, tools) 174 | 175 | return agent 176 | 177 | 178 | 179 | # Initialize Faker 180 | fake = Faker() 181 | 182 | # Get today's date 183 | today = datetime.now().date() 184 | 185 | # City data with coordinates (latitude, longitude) 186 | city_data = { 187 | "New York": (40.7128, -74.0060), 188 | "Los Angeles": (34.0522, -118.2437), 189 | "Chicago": (41.8781, -87.6298), 190 | "Las Vegas": (36.1699, -115.1398), 191 | "San Francisco": (37.7749, -122.4194), 192 | "Orlando": (28.5383, -81.3792), 193 | "Miami": (25.7617, -80.1918), 194 | "Washington": (38.9072, -77.0369), 195 | "New Orleans": (29.9511, -90.0715), 196 | "Boston": (42.3601, -71.0589), 197 | "Seattle": (47.6062, -122.3321), 198 | "San Diego": (32.7157, -117.1611), 199 | "Honolulu": (21.3069, -157.8583), 200 | "Nashville": (36.1627, -86.7816), 201 | "Denver": (39.7392, -104.9903), 202 | "Austin": (30.2672, -97.7431), 203 | "Philadelphia": (39.9526, -75.1652), 204 | "Atlanta": (33.7490, -84.3880), 205 | "San Antonio": (29.4241, -98.4936), 206 | "Portland": (45.5155, -122.6789), 207 | "Paris": (48.8566, 2.3522), 208 | "London": (51.5074, -0.1278), 209 | "Rome": (41.9028, 12.4964), 210 | "Barcelona": (41.3851, 2.1734), 211 | "Amsterdam": (52.3676, 4.9041), 212 | "Berlin": (52.5200, 13.4050), 213 | "Vienna": (48.2082, 16.3738), 214 | "Prague": (50.0755, 14.4378), 215 | "Venice": (45.4408, 12.3155), 216 | "Florence": (43.7696, 11.2558), 217 | "Istanbul": (41.0082, 28.9784), 218 | "Madrid": (40.4168, -3.7038), 219 | "Lisbon": (38.7223, -9.1393), 220 | "Dublin": (53.3498, -6.2603), 221 | "Budapest": (47.4979, 19.0402), 222 | "Athens": (37.9838, 23.7275), 223 | "Edinburgh": (55.9533, -3.1883), 224 | "Copenhagen": (55.6761, 12.5683), 225 | "Stockholm": (59.3293, 18.0686), 226 | "Brussels": (50.8503, 4.3517), 227 | "Zurich": (47.3769, 8.5417), 228 | "Milan": (45.4642, 9.1900), 229 | "Munich": (48.1351, 11.5820), 230 | "Seville": (37.3891, -5.9845), 231 | } 232 | 233 | def calculate_distance(lat1, lon1, lat2, lon2): 234 | R = 6371 # Earth's radius in kilometers 235 | 236 | lat1, lon1, lat2, lon2 = map(math.radians, [lat1, lon1, lat2, lon2]) 237 | dlat = lat2 - lat1 238 | dlon = lon2 - lon1 239 | 240 | a = ( 241 | math.sin(dlat / 2) ** 2 242 | + math.cos(lat1) * math.cos(lat2) * math.sin(dlon / 2) ** 2 243 | ) 244 | c = 2 * math.asin(math.sqrt(a)) 245 | distance = R * c 246 | 247 | return distance 248 | 249 | def calculate_flight_duration(distance): 250 | # Assume average speed of 800 km/h and add 30 minutes for takeoff and landing 251 | duration_hours = distance / 800 252 | duration_minutes = int(duration_hours * 60) + 30 253 | return timedelta(minutes=duration_minutes) 254 | 255 | def calculate_flight_price(distance): 256 | # Base price of $50 plus $0.1 per km 257 | base_price = 50 258 | price_per_km = 0.1 259 | return round(base_price + (distance * price_per_km), 2) 260 | 261 | def generate_user(user_id): 262 | name = fake.name() 263 | age = random.randint(18, 80) 264 | home_location = random.choice(list(city_data.keys())) 265 | 266 | return (user_id, name, age, home_location) 267 | 268 | def generate_flight_booking(booking_id, user_id, user_name): 269 | origin = random.choice(list(city_data.keys())) 270 | destination = random.choice([city for city in city_data.keys() if city != origin]) 271 | 272 | origin_coords = city_data[origin] 273 | dest_coords = city_data[destination] 274 | 275 | distance = calculate_distance( 276 | origin_coords[0], origin_coords[1], dest_coords[0], dest_coords[1] 277 | ) 278 | flight_duration = calculate_flight_duration(distance) 279 | price = calculate_flight_price(distance) 280 | 281 | # Generate a random booking date within the next 15 days 282 | booking_date = today + timedelta(days=random.randint(0, 14)) 283 | 284 | # Generate departure date between 1 and 30 days after the booking date 285 | departure_date = booking_date + timedelta(days=random.randint(1, 30)) 286 | 287 | # Generate departure time 288 | departure_time = f"{random.randint(0, 23):02d}:{random.randint(0, 59):02d}" 289 | 290 | # Calculate arrival date and time 291 | departure_datetime = datetime.combine( 292 | departure_date, datetime.strptime(departure_time, "%H:%M").time() 293 | ) 294 | arrival_datetime = departure_datetime + flight_duration 295 | 296 | return ( 297 | booking_id, 298 | user_id, 299 | user_name, 300 | origin, 301 | destination, 302 | price, 303 | int(flight_duration.total_seconds() // 60), # Convert to minutes 304 | departure_date.strftime("%Y-%m-%d"), 305 | departure_time, 306 | arrival_datetime.strftime("%Y-%m-%d"), 307 | arrival_datetime.strftime("%H:%M"), 308 | round(distance, 2), 309 | booking_date.strftime("%Y-%m-%d"), 310 | ) 311 | 312 | def generate_hotel_booking(booking_id, user_id, user_name): 313 | city = random.choice(list(city_data.keys())) 314 | hotel_name = fake.company() + " Hotel" 315 | check_in_date = fake.date_between(start_date="+1d", end_date="+60d") 316 | nights = random.randint(1, 14) 317 | check_out_date = check_in_date + timedelta(days=nights) 318 | price_per_night = round(random.uniform(50, 500), 2) 319 | total_price = round(price_per_night * nights, 2) 320 | num_guests = random.randint(1, 4) 321 | room_type = random.choice(["Single", "Double", "Suite", "Deluxe"]) 322 | 323 | return ( 324 | booking_id, 325 | user_id, 326 | user_name, 327 | city, 328 | hotel_name, 329 | check_in_date.strftime("%Y-%m-%d"), 330 | check_out_date.strftime("%Y-%m-%d"), 331 | nights, 332 | price_per_night, 333 | total_price, 334 | num_guests, 335 | room_type, 336 | ) 337 | 338 | def create_database(): 339 | try: 340 | os.remove("data/travel_bookings.db") 341 | except OSError: 342 | pass 343 | conn = sqlite3.connect("data/travel_bookings.db") 344 | cursor = conn.cursor() 345 | 346 | cursor.execute( 347 | """ 348 | CREATE TABLE IF NOT EXISTS users ( 349 | user_id INTEGER PRIMARY KEY, 350 | name TEXT, 351 | age INTEGER, 352 | home_location TEXT 353 | ) 354 | """ 355 | ) 356 | 357 | cursor.execute( 358 | """ 359 | CREATE TABLE IF NOT EXISTS flight_bookings ( 360 | booking_id INTEGER PRIMARY KEY, 361 | user_id INTEGER, 362 | user_name TEXT, 363 | origin TEXT, 364 | destination TEXT, 365 | price REAL, 366 | flight_duration INTEGER, 367 | departure_date TEXT, 368 | departure_time TEXT, 369 | arrival_date TEXT, 370 | arrival_time TEXT, 371 | distance REAL, 372 | booking_date TEXT, 373 | FOREIGN KEY (user_id) REFERENCES users (user_id) 374 | ) 375 | """ 376 | ) 377 | 378 | cursor.execute( 379 | """ 380 | CREATE TABLE IF NOT EXISTS hotel_bookings ( 381 | booking_id INTEGER PRIMARY KEY, 382 | user_id INTEGER, 383 | user_name TEXT, 384 | city TEXT, 385 | hotel_name TEXT, 386 | check_in_date TEXT, 387 | check_out_date TEXT, 388 | nights INTEGER, 389 | price_per_night REAL, 390 | total_price REAL, 391 | num_guests INTEGER, 392 | room_type TEXT, 393 | FOREIGN KEY (user_id) REFERENCES users (user_id) 394 | ) 395 | """ 396 | ) 397 | 398 | conn.commit() 399 | return conn, cursor 400 | 401 | def insert_sample_data( 402 | conn, cursor, num_users, num_flight_bookings, num_hotel_bookings 403 | ): 404 | # Generate and insert user data 405 | users = [generate_user(i + 1) for i in range(num_users)] 406 | cursor.executemany( 407 | """ 408 | INSERT INTO users (user_id, name, age, home_location) 409 | VALUES (?, ?, ?, ?) 410 | """, 411 | users, 412 | ) 413 | 414 | # Generate and insert flight bookings 415 | for i in range(num_flight_bookings): 416 | user = random.choice(users) 417 | flight_booking = generate_flight_booking(i + 1, user[0], user[1]) 418 | cursor.execute( 419 | """ 420 | INSERT INTO flight_bookings (booking_id, user_id, user_name, origin, destination, price, flight_duration, departure_date, departure_time, arrival_date, arrival_time, distance, booking_date) 421 | VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) 422 | """, 423 | flight_booking, 424 | ) 425 | 426 | # Generate and insert hotel bookings 427 | for i in range(num_hotel_bookings): 428 | user = random.choice(users) 429 | hotel_booking = generate_hotel_booking(i + 1, user[0], user[1]) 430 | cursor.execute( 431 | """ 432 | INSERT INTO hotel_bookings (booking_id, user_id, user_name, city, hotel_name, check_in_date, check_out_date, nights, price_per_night, total_price, num_guests, room_type) 433 | VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) 434 | """, 435 | hotel_booking, 436 | ) 437 | 438 | conn.commit() 439 | 440 | if __name__ == "__main__": 441 | conn, cursor = create_database() 442 | insert_sample_data( 443 | conn, cursor, num_users=500, num_flight_bookings=1000, num_hotel_bookings=1000 444 | ) 445 | conn.close() 446 | print("Sample data has been generated and inserted into the SQLite database.") 447 | -------------------------------------------------------------------------------- /07_Cross_Region_Inference/README.md: -------------------------------------------------------------------------------- 1 | # Scalable inference with Amazon Bedrock 2 | Amazon Bedrock provides easy access to leading foundation models from multiple model providers. Bedrock offers various inference mechanisms to consume these models. Following different types are available in Bedrock: 3 | * On-demand inference – The standard option for inference. Involves invoking a model in a specific AWS Region. 4 | * Cross-region inference – The opt-in feature allowing access to foundation models beyond a single region. It involves invoking an inference profile, an abstraction over foundation models deployed across a set of AWS Regions, to which it can route inference requests. It helps in increasing throughput and improves resiliency by dynamically routing requests (to the regions defined in the inference profile), depending on user traffic and demand. 5 | * Provisioned Throughput – The option to purchase dedicated compute. It involves purchasing a dedicated level of throughput for a model in a specific AWS Region. Provisioned Throughput quotas depend on the number of model units that you purchase. 6 | 7 | 8 | ## Contents 9 | This folder contains examples to get you started with various inference patterns for Amazon Bedrock. Choose from the examples below to learn more: 10 | 1. [Getting Started with Cross-region Inference](Getting_started_with_Cross-region_Inference.ipynb) - Introductory sample to help you understand how to use cross-region inference feature from Amazon Bedrock. 11 | 12 | 13 | ## Contributing 14 | 15 | We welcome community contributions! Please ensure your sample aligns with AWS [best practices](https://aws.amazon.com/architecture/well-architected/), and please update the **Contents** section of this README file with a link to your sample, along with a description. 16 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Amazon Bedrock Workshop [![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/dwyl/esta/issues) 2 | 3 | This hands-on workshop, aimed at developers and solution builders, introduces how to leverage foundation models (FMs) through [Amazon Bedrock](https://aws.amazon.com/bedrock/). This code goes alongside the self-paced or instructor lead workshop here - https://catalog.us-east-1.prod.workshops.aws/amazon-bedrock/en-US 4 | 5 | **Please follow the prerequisites listed in the link above or ask your AWS workshop instructor how to get started.** 6 | 7 | Amazon Bedrock is a fully managed service that provides access to FMs from third-party providers and Amazon; available via an API. With Bedrock, you can choose from a variety of models to find the one that’s best suited for your use case. 8 | 9 | Within this series of labs, you'll explore some of the most common usage patterns we are seeing with our customers for Generative AI. We will show techniques for generating text and images, creating value for organizations by improving productivity. This is achieved by leveraging foundation models to help in composing emails, summarizing text, answering questions, building chatbots, and creating images. While the focus of this workshop is for you to gain hands-on experience implementing these patterns via Bedrock APIs and SDKs, you will also have the option of exploring integrations with open-source packages like [LangChain](https://python.langchain.com/docs/get_started/introduction) and [FAISS](https://faiss.ai/index.html). 10 | 11 | Labs include: 12 | 13 | - **01 - Text Generation** \[Estimated time to complete - 25 mins\] [![Test - pass](https://img.shields.io/badge/Test-pass-2ea44f)](https://) 14 | - Text, code generation with Bedrock 15 | - **02 - Knowledge bases and RAG** \[Estimated time to complete - 35 mins\] [![Test - pass](https://img.shields.io/badge/Test-pass-2ea44f)](https://) 16 | - Managed RAG retrieve and generate example 17 | - Langchain RAG retrieve and generate example 18 | - **03 - Model customization** \[Estimated time to complete - 30 mins\] [![Test - pass](https://img.shields.io/badge/Test-pass-2ea44f)](https://) 19 | - Fine tuning Titan lite, Llama2 20 | - **Note** - _You must run this on your own AWS account, and this will not work on AWS Workshop Studio!_ 21 | - **04 - Image and Multimodal** \[Estimated time to complete - 30 mins\] [![Test - pass](https://img.shields.io/badge/Test-pass-2ea44f)](https://) 22 | - Bedrock Titan image generator 23 | - Bedrock Stable Diffusion XL 24 | - Bedrock Titan Multimodal embeddings 25 | - Nova Reel and Canvas notebooks 26 | - **05 - Agents** \[Estimated time to complete - 30 mins\] [![Test - pass](https://img.shields.io/badge/Test-pass-2ea44f)](https://) 27 | - Customer service agent 28 | - Insurance claims agent 29 | - **06 - Open source examples (optional)** \[Estimated time to complete - 30 mins\] [![Test - fail](https://img.shields.io/badge/Test-fail-red)](https://) 30 | - Langchain Text Generation examples 31 | - Langchain KB RAG examples 32 | - Langchain Chatbot examples 33 | - NVIDIA NeMo Guardrails examples 34 | - NodeJS Bedrock examples 35 | 36 |
37 | 38 | ![imgs/11-overview](imgs/11-overview.png "Overview of the different labs in the workshop") 39 | 40 |
41 | 42 | You can also refer to these [Step-by-step guided instructions on the workshop website](https://catalog.us-east-1.prod.workshops.aws/workshops/a4bdb007-5600-4368-81c5-ff5b4154f518/en-US). 43 | 44 | 45 | ## Getting started 46 | 47 | ### Choose a notebook environment 48 | 49 | This workshop is presented as a series of **Python notebooks**, which you can run from the environment of your choice: 50 | 51 | - For a fully-managed environment with rich AI/ML features, we'd recommend using [SageMaker Studio](https://aws.amazon.com/sagemaker/studio/). To get started quickly, you can refer to the [instructions for domain quick setup](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html). 52 | - For a fully-managed but more basic experience, you could instead [create a SageMaker Notebook Instance](https://docs.aws.amazon.com/sagemaker/latest/dg/howitworks-create-ws.html). 53 | - If you prefer to use your existing (local or other) notebook environment, make sure it has [credentials for calling AWS](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). 54 | 55 | 56 | ### Enable AWS IAM permissions for Bedrock 57 | 58 | The AWS identity you assume from your notebook environment (which is the [*Studio/notebook Execution Role*](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) from SageMaker, or could be a role or IAM User for self-managed notebooks), must have sufficient [AWS IAM permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html) to call the Amazon Bedrock service. 59 | 60 | To grant Bedrock access to your identity, you can: 61 | 62 | - Open the [AWS IAM Console](https://us-east-1.console.aws.amazon.com/iam/home?#) 63 | - Find your [Role](https://us-east-1.console.aws.amazon.com/iamv2/home?#/roles) (if using SageMaker or otherwise assuming an IAM Role), or else [User](https://us-east-1.console.aws.amazon.com/iamv2/home?#/users) 64 | - Select *Add Permissions > Create Inline Policy* to attach new inline permissions, open the *JSON* editor and paste in the below example policy: 65 | 66 | ``` 67 | { 68 | "Version": "2012-10-17", 69 | "Statement": [ 70 | { 71 | "Sid": "BedrockFullAccess", 72 | "Effect": "Allow", 73 | "Action": ["bedrock:*"], 74 | "Resource": "*" 75 | } 76 | ] 77 | } 78 | ``` 79 | 80 | > ⚠️ **Note:** With Amazon SageMaker, your notebook execution role will typically be *separate* from the user or role that you log in to the AWS Console with. If you'd like to explore the AWS Console for Amazon Bedrock, you'll need to grant permissions to your Console user/role too. You can run the notebooks anywhere as long as you have access to the AWS Bedrock service and have appropriate credentials 81 | 82 | For more information on the fine-grained action and resource permissions in Bedrock, check out the Bedrock Developer Guide. 83 | 84 | 85 | ### Clone and use the notebooks 86 | 87 | > ℹ️ **Note:** In SageMaker Studio, you can open a "System Terminal" to run these commands by clicking *File > New > Terminal* 88 | 89 | Once your notebook environment is set up, clone this workshop repository into it. 90 | 91 | ```sh 92 | sudo yum install -y unzip 93 | git clone https://github.com/aws-samples/amazon-bedrock-workshop.git 94 | cd amazon-bedrock-workshop 95 | ``` 96 | 97 | [![HitCount](https://hits.dwyl.com/aws-samples/amazon-bedrock-workshop.svg?style=flat-square&show=unique)](http://hits.dwyl.com/aws-samples/amazon-bedrock-workshop) 98 | 99 | 100 | ## Star History 101 | 102 | [![Star History Chart](https://api.star-history.com/svg?repos=aws-samples/amazon-bedrock-workshop&type=Date)](https://star-history.com/#aws-samples/amazon-bedrock-workshop&Date) 103 | 104 | # 👥 Contributors 105 | 106 | Thanks to our awesome contributors! 🚀🚀🚀 107 | 108 | [![contributors](https://contrib.rocks/image?repo=aws-samples/amazon-bedrock-workshop&max=2000)](https://github.com/aws-samples/amazon-bedrock-workshop/graphs/contributors) 109 | -------------------------------------------------------------------------------- /RELEASE_NOTES.md: -------------------------------------------------------------------------------- 1 | ### 4/15/2025 2 | - remove module 0 (prerequisites) 3 | - full refactor of module 1 4 | - update to newer models 5 | - update markdown instructions and formatting 6 | - update other module assets 7 | - discuss new topics - Converse API, CRIS, function calling, etc. 8 | - full refactor of module 2 9 | - update to newer models 10 | - update markdown instructions and formatting 11 | - rename notebooks 12 | - refresh of module 4 13 | - update markdown formatting 14 | - refresh of module 5 15 | - notebook renaming / re-ordering 16 | 17 | 18 | ### 1/5/2024 19 | - Added Nova canvas notebook 20 | - Added Nova Reel notebook 21 | - Added inline agents 22 | 23 | 24 | ### 8/8/2024 25 | 26 | - New (cool) shortlink for you to remember and share! https://catalog.workshops.aws/amazon-bedrock :blob-cat-party: 27 | 28 | - Bedrock Workshop will now be run in the new JupyterLab on SageMaker Studio and not classic Studio notebooks 29 | 30 | - Instructions and Prereq’s have been changed in the pinned build (to be published) ... *Changes go live on 8/15* :alert: 31 | 32 | - Weekly automated testing to make sure all labs run successfully! 33 | 34 | ### 5/20/2024 35 | - rolled back TTP to Titan Large in text generation 36 | - rolled back Claude 3 to 2.1 in basic testing of RAG folder (0_ notebook) 37 | - fixed utility file saying "Done!" every 30 seconds. Now it just does the dots. Can improve further but this should work for now. 38 | - fixed %store etc. issues in RAG folder 39 | 40 | ### 5/10/2024 41 | - changed pre-req notebook 42 | - Titan text premier (TTP) tested and works 43 | - Titan emb v2 tested and works 44 | - claude Haiku v2 instead of v2 in the messages API 45 | - changed few 01_Text_generation notebooks: 46 | - 00 now has both Claude and TTP 47 | - 01 now uses TTP 48 | - 02 now uses Claude and TTP examples 49 | - 03, depending on the model can answer even though for demonstration purposes we want the model to say it is unsure. Sticking to Titan Large for this, for now. 50 | - 04 changed readme to encourage customers to try TTP. 51 | 52 | ### 2/15/2024 53 | 54 | - new 'basic' set up and testing which removes the need for bedrock utils folder. All notebooks now just use default boto3 bedrock/bedrock-runtime 55 | - Some notebooks have moved as is, and others have been merged. Making sure instructions in the notebook are still correct (markdown sections) is important 56 | - Open source examples (Langchain, Nemo) have been moved to a separate folder. Some existing PRs can be fixed and tested directly here. Once we test it we can resolve those PRs and point to the new release. 57 | - Fine tuning and other new feature examples are needed 58 | - Pending Major PRs 59 | - https://github.com/aws-samples/amazon-bedrock-workshop/pull/194 60 | - https://github.com/aws-samples/amazon-bedrock-workshop/pull/187 61 | - https://github.com/aws-samples/amazon-bedrock-workshop/pull/149 62 | - https://github.com/aws-samples/amazon-bedrock-workshop/pull/107 63 | 64 | 65 | ### 2/10/2024 66 | - Major structural changes 67 | - working branch is BR-workshop-v2 68 | -------------------------------------------------------------------------------- /imgs/10-overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/10-overview.png -------------------------------------------------------------------------------- /imgs/11-overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/11-overview.png -------------------------------------------------------------------------------- /imgs/41-text-simple-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/41-text-simple-1.png -------------------------------------------------------------------------------- /imgs/42-text-summarization-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/42-text-summarization-2.png -------------------------------------------------------------------------------- /imgs/51-simple-rag.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/51-simple-rag.png -------------------------------------------------------------------------------- /imgs/52-rag-with-external-data.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/52-rag-with-external-data.png -------------------------------------------------------------------------------- /imgs/53-rag-with-pinecone.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/53-rag-with-pinecone.png -------------------------------------------------------------------------------- /imgs/Chatbot_lang.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/Chatbot_lang.png -------------------------------------------------------------------------------- /imgs/Embeddings_lang (1).png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/Embeddings_lang (1).png -------------------------------------------------------------------------------- /imgs/Embeddings_pinecone_lang.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/Embeddings_pinecone_lang.png -------------------------------------------------------------------------------- /imgs/bedrock-code-gen.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/bedrock-code-gen.png -------------------------------------------------------------------------------- /imgs/chatbot_bedrock.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/chatbot_bedrock.png -------------------------------------------------------------------------------- /imgs/chatbot_lang.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/chatbot_lang.png -------------------------------------------------------------------------------- /imgs/context-aware-chatbot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/context-aware-chatbot.png -------------------------------------------------------------------------------- /imgs/embeddings_lang.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/embeddings_lang.png -------------------------------------------------------------------------------- /imgs/logic-overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-bedrock-workshop/6a6198c4c27bddc1699aa0975d275ffee0250b15/imgs/logic-overview.png --------------------------------------------------------------------------------