├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── aws-creds.py ├── frontend ├── .gitignore ├── README.md ├── babel.config.js ├── package.json ├── parameters.json ├── public │ └── index.html └── src │ ├── App.vue │ ├── app.css │ ├── aws-exports.js │ ├── components │ └── NotFoundPage.vue │ ├── main.js │ ├── router.js │ ├── utils │ ├── auth.js │ └── core.js │ └── views │ ├── AboutView.vue │ ├── Home.vue │ ├── KB.vue │ ├── LLMs.vue │ ├── LoginView.vue │ ├── PromptView.vue │ └── RAG.vue ├── lambdas ├── bedrockFunc │ ├── cognitouser.py │ └── requirements.txt ├── llmFunctions │ ├── kbfunction.py │ └── llmfunction.py └── ragFunctions │ ├── promptfunction.py │ ├── ragfunction.py │ └── requirements.txt ├── prompt-engineering ├── claude-prompt-template.txt ├── llama-prompt-template.txt └── mistral-prompt-template.txt ├── startup.sh ├── template.yaml └── tools ├── account-setup.sh ├── gen-ai-builder-init v1.1.yml ├── gen-ai-builder-init-v1.2.yaml ├── resize.sh ├── samconf.toml ├── solutions ├── kbfunction.py ├── llmfunction.py └── ragfunction.py └── troubleshoot.sh /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | .DS_Store 3 | *.toml 4 | .aws-sam/ 5 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Amazon Bedrock Serverless Workshop 2 | 3 | This project contains source code and supporting files for a serverless application that you can deploy Amazon Bedrock Serverless solution using the SAM CLI. In this, you use the Retrieval-Augmented Generation (RAG) technique to build a generative AI-powered chatbot. A foundation model (FM) in Amazon Bedrock is used to answer your chatbot questions through pre-indexed content. Amazon Bedrock is a fully managed service that makes FMs (from Amazon and leading AI startups) available through an API, so you can choose from various FMs to find the model that's best suited to your use case. For storing (indexing) and retrieving relevant content, you use Amazon Kendra, a fully managed service that provides intelligent enterprise searches powered by machine learning. You use AWS Lambda as the serverless compute for running the application code in an event-driven manner. 4 | 5 | The project includes the following files and folders. 6 | - **Bedrock Function:** Code for the application's RAG Lambda function, and also a function for authenticating the app. 7 | - **UI code:** The code is developed using Amazon Amplify with Vue JavaScript framework. 8 | - **Sam Template:** A template (template.yaml) that defines the application's AWS backend resources. Once you run this using SAM, it creates Amazon API Gateway endpoint, AWS Lambda function (RAG), and a Amazon Cognito user pool. 9 | 10 | ## Deploy and test Amazon Bedrock serverless application 11 | 12 | To deploy and test this code, visit the following Amazon Bedrock workshop. 13 | 14 | See the [Creating a Serverless Chatbot using Amazon Bedrock](https://catalog.us-east-1.prod.workshops.aws/workshops/27eb3134-4f33-4689-bb73-269e4273947a) 15 | 16 | ## Resources 17 | 18 | - [AWS SAM developer guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html) 19 | - [Amazon Kendra developer guide](https://docs.aws.amazon.com/kendra/latest/dg/what-is-kendra.html) 20 | - [Amazon Bedrock user guide](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) 21 | 22 | ## Security 23 | 24 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 25 | 26 | ## License 27 | 28 | This library is licensed under the MIT-0 License. See the LICENSE file. 29 | 30 | -------------------------------------------------------------------------------- /aws-creds.py: -------------------------------------------------------------------------------- 1 | import os 2 | import boto3 3 | 4 | # Get the current AWS credentials 5 | credentials = boto3.Session().get_credentials() 6 | 7 | # Create the AWS credentials directory if it doesn't exist 8 | aws_dir = os.path.expanduser('~/.aws') 9 | if not os.path.exists(aws_dir): 10 | os.makedirs(aws_dir) 11 | 12 | # Write the credentials to the AWS credentials file 13 | credential_file = os.path.join(aws_dir, 'credentials') 14 | with open(credential_file, 'w') as f: 15 | f.write('[default]\n') 16 | f.write('region=us-west-2\n') 17 | f.write(f'accessKeyId={credentials.access_key}\n') 18 | f.write(f'secretAccessKey={credentials.secret_key}\n') 19 | if credentials.token: 20 | f.write(f'sessionToken={credentials.token}\n') -------------------------------------------------------------------------------- /frontend/.gitignore: -------------------------------------------------------------------------------- 1 | 2 | 3 | #amplify-do-not-edit-begin 4 | amplify/\#current-cloud-backend 5 | amplify/.config/local-* 6 | amplify/logs 7 | amplify/mock-data 8 | amplify/mock-api-resources 9 | amplify/backend/amplify-meta.json 10 | amplify/backend/.temp 11 | build/ 12 | dist/ 13 | node_modules/ 14 | aws-exports.js 15 | awsconfiguration.json 16 | amplifyconfiguration.json 17 | amplifyconfiguration.dart 18 | amplify-build-config.json 19 | amplify-gradle-config.json 20 | amplifytools.xcconfig 21 | .secret-* 22 | **.sample 23 | #amplify-do-not-edit-end 24 | -------------------------------------------------------------------------------- /frontend/README.md: -------------------------------------------------------------------------------- 1 | # Tasks webapp 2 | 3 | ## Project setup 4 | 5 | ```sh 6 | npm install 7 | ``` 8 | 9 | ## Configure API Gateway endpoint URL 10 | 11 | Edit `src/main.js` and set the value for `axios.defaults.baseURL` to your API Gateway endpoint. This value is output from the SAM template in the sibling project to this one. 12 | 13 | ### Compiles and hot-reloads for development 14 | 15 | ```sh 16 | npm run serve 17 | ``` 18 | 19 | ### Compiles and minifies for production 20 | 21 | ```sh 22 | npm run build 23 | ``` 24 | 25 | ### Install latest version of Amplify CLI 26 | 27 | ```sh 28 | npm i -g @aws-amplify/cli 29 | ``` 30 | 31 | ### Initialize Amplify 32 | 33 | ```sh 34 | amplify init 35 | ``` 36 | 37 | ### Add hosting 38 | 39 | ```sh 40 | amplify add hosting 41 | ``` 42 | 43 | ### Publish app 44 | 45 | ```sh 46 | amplify publish 47 | ``` -------------------------------------------------------------------------------- /frontend/babel.config.js: -------------------------------------------------------------------------------- 1 | module.exports = { 2 | presets: [ 3 | '@vue/cli-plugin-babel/preset' 4 | ] 5 | } 6 | -------------------------------------------------------------------------------- /frontend/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "getting-started-serverless", 3 | "version": "0.1.0", 4 | "private": true, 5 | "scripts": { 6 | "serve": "vue-cli-service serve", 7 | "build": "vue-cli-service build", 8 | "lint": "vue-cli-service lint" 9 | }, 10 | "dependencies": { 11 | "@aws-sdk/client-cognito-identity-provider": "^3.418.0", 12 | "amazon-cognito-identity-js": "^6.3.5", 13 | "axios": "^0.27.2", 14 | "babel-preset-expo": "^9.5.2", 15 | "core-js": "^3.19.1", 16 | "electron": "^26.2.0", 17 | "jwt-decode": "^3.1.2", 18 | "linkify-html": "^4.1.1", 19 | "linkifyjs": "^4.1.1", 20 | "util": "^0.12.5", 21 | "vue": "^2.6.14", 22 | "vue-axios": "^3.5.2", 23 | "vue-router": "^3.5.3", 24 | "webpack": "^4.46.0" 25 | }, 26 | "devDependencies": { 27 | "@expo/webpack-config": "^18.1.3", 28 | "@vue/cli-plugin-babel": "~5.0.8", 29 | "@vue/cli-plugin-eslint": "~5.0.8", 30 | "@vue/cli-service": "~5.0.8", 31 | "babel-eslint": "^10.1.0", 32 | "eslint": "^7.32.0", 33 | "eslint-plugin-vue": "^8.0.3", 34 | "vue-template-compiler": "^2.6.14" 35 | }, 36 | "eslintConfig": { 37 | "root": true, 38 | "env": { 39 | "node": true 40 | }, 41 | "extends": [ 42 | "plugin:vue/essential", 43 | "eslint:recommended" 44 | ], 45 | "parserOptions": { 46 | "parser": "babel-eslint" 47 | }, 48 | "rules": {} 49 | }, 50 | "browserslist": [ 51 | "> 1%", 52 | "last 2 versions", 53 | "not dead" 54 | ] 55 | } 56 | -------------------------------------------------------------------------------- /frontend/parameters.json: -------------------------------------------------------------------------------- 1 | { 2 | "plugin": "Hosting with Amplify Console (Managed hosting with custom domains, Continuous deployment)", 3 | "type": "Manual deployment" 4 | } -------------------------------------------------------------------------------- /frontend/public/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 11 | 12 | Bedrock Workshop 13 | 14 | 15 | 16 | 20 |
21 | 22 | 23 | 26 | 29 | 32 | 33 | 34 | 35 | -------------------------------------------------------------------------------- /frontend/src/App.vue: -------------------------------------------------------------------------------- 1 | 56 | 57 | 79 | 80 | -------------------------------------------------------------------------------- /frontend/src/app.css: -------------------------------------------------------------------------------- 1 | *, 2 | *::before, 3 | *::after { 4 | box-sizing: border-box; 5 | } 6 | 7 | :root { 8 | --select-border: #777; 9 | --select-focus: blue; 10 | --select-arrow: var(--select-border); 11 | } 12 | 13 | .output pre { 14 | white-space: pre-wrap; 15 | word-wrap: break-word; 16 | font-family: inherit; 17 | } 18 | 19 | select { 20 | -webkit-appearance: none; 21 | -moz-appearance: none; 22 | appearance: none; 23 | background-color: transparent; 24 | border: none; 25 | padding: 0 1em 0 0; 26 | margin: 0; 27 | width: 100%; 28 | font-family: inherit; 29 | font-size: inherit; 30 | cursor: inherit; 31 | line-height: inherit; 32 | z-index: 1; 33 | outline: none; 34 | } 35 | select::-ms-expand { 36 | display: none; 37 | } 38 | 39 | .select { 40 | display: grid; 41 | grid-template-areas: "select"; 42 | align-items: center; 43 | position: relative; 44 | min-width: 15ch; 45 | max-width: 30ch; 46 | border: 1px solid var(--select-border); 47 | border-radius: 0.25em; 48 | padding: 0.25em 0.5em; 49 | font-size: 1rem; 50 | cursor: pointer; 51 | line-height: 1.1; 52 | background-color: #fff; 53 | background-image: linear-gradient(to top, #f9f9f9, #fff 33%); 54 | } 55 | .select select, .select::after { 56 | grid-area: select; 57 | } 58 | .select:not(.select--multiple)::after { 59 | content: ""; 60 | justify-self: end; 61 | width: 0.8em; 62 | height: 0.5em; 63 | background-color: var(--select-arrow); 64 | -webkit-clip-path: polygon(100% 0%, 0 0%, 50% 100%); 65 | clip-path: polygon(100% 0%, 0 0%, 50% 100%); 66 | } 67 | 68 | select:focus + .focus { 69 | position: absolute; 70 | top: -1px; 71 | left: -1px; 72 | right: -1px; 73 | bottom: -1px; 74 | border: 2px solid var(--select-focus); 75 | border-radius: inherit; 76 | } 77 | 78 | select[multiple] { 79 | padding-right: 0; 80 | /* 81 | * Safari will not reveal an option 82 | * unless the select height has room to 83 | * show all of it 84 | * Firefox and Chrome allow showing 85 | * a partial option 86 | */ 87 | height: 6rem; 88 | /* 89 | * Experimental - styling of selected options 90 | * in the multiselect 91 | * Not supported crossbrowser 92 | */ 93 | } 94 | select[multiple] option { 95 | white-space: normal; 96 | outline-color: var(--select-focus); 97 | } 98 | 99 | .select--disabled { 100 | cursor: not-allowed; 101 | background-color: #eee; 102 | background-image: linear-gradient(to top, #ddd, #eee 33%); 103 | } 104 | 105 | label { 106 | font-size: 1rem; 107 | font-weight: 500; 108 | } 109 | 110 | .select + label { 111 | margin-top: 1rem; 112 | } 113 | 114 | body { 115 | min-height: 100vh; 116 | display: grid; 117 | 118 | grid-gap: 0.5rem; 119 | font-family: "Baloo 2", sans-serif; 120 | background-color: #e9f2fd; 121 | 122 | } 123 | -------------------------------------------------------------------------------- /frontend/src/aws-exports.js: -------------------------------------------------------------------------------- 1 | /* eslint-disable */ 2 | // WARNING: DO NOT EDIT. This file is automatically generated by AWS Amplify. It will be overwritten. 3 | 4 | const awsmobile = { 5 | "aws_project_region": "us-west-2" 6 | }; 7 | 8 | 9 | export default awsmobile; 10 | -------------------------------------------------------------------------------- /frontend/src/components/NotFoundPage.vue: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /frontend/src/main.js: -------------------------------------------------------------------------------- 1 | import Vue from 'vue' 2 | import App from './App.vue' 3 | import axios from 'axios' 4 | import router from './router' 5 | import VueAxios from 'vue-axios' 6 | 7 | Vue.config.productionTip = false 8 | 9 | //These values are replaces during amplify setup 10 | axios.defaults.baseURL = "" 11 | Vue.prototype.$UserPoolId = '' 12 | Vue.prototype.$ClientId = '' 13 | 14 | Vue.use(VueAxios, axios) 15 | 16 | Vue.config.productionTip = false 17 | 18 | new Vue({ 19 | router, 20 | render: h => h(App), 21 | 22 | }).$mount('#app') 23 | -------------------------------------------------------------------------------- /frontend/src/router.js: -------------------------------------------------------------------------------- 1 | import Vue from 'vue' 2 | import Router from 'vue-router' 3 | import PromptView from './views/PromptView.vue' 4 | import RAG from './views/RAG.vue' 5 | import LLMs from './views/LLMs.vue' 6 | import LoginView from './views/LoginView.vue' 7 | import AboutView from './views/AboutView.vue' 8 | import NotFoundPage from './components/NotFoundPage.vue' 9 | import KB from './views/KB.vue' 10 | import { clearAuthToken, isLoggedIn } from './utils/auth' 11 | 12 | Vue.use(Router) 13 | 14 | const router = new Router({ 15 | mode: 'history', 16 | base: process.env.BASE_URL, 17 | routes: [ 18 | { 19 | path: '/', 20 | name: 'llms', 21 | component: LLMs 22 | }, 23 | { 24 | path: '/rag', 25 | name: 'rag', 26 | component: RAG 27 | }, 28 | { 29 | path: '/kb', 30 | name: 'kb', 31 | component: KB 32 | }, 33 | { 34 | path: '/prompt', 35 | name: 'prompt', 36 | component: PromptView 37 | }, 38 | { 39 | path: '/login', 40 | name: 'login', 41 | component: LoginView, 42 | meta: { 43 | allowAnonymous: true 44 | } 45 | }, 46 | { 47 | path: '/logout', 48 | name: 'Logout', 49 | beforeEnter(to, from, next) { 50 | clearAuthToken() 51 | next('/login') 52 | }, 53 | }, 54 | { 55 | path: '/about', 56 | name: 'about', 57 | component: AboutView 58 | }, 59 | { 60 | path: '*', 61 | component: NotFoundPage 62 | } 63 | ] 64 | }) 65 | 66 | 67 | router.beforeEach((to, from, next) => { 68 | 69 | 70 | if (to.name == 'login' && isLoggedIn()) { 71 | console.log('redirecting to /') 72 | next({ path: '/' }) 73 | } 74 | else if (!to.meta.allowAnonymous && !isLoggedIn()) { 75 | 76 | next({ 77 | path: '/login', 78 | query: { redirect: to.fullPath } 79 | }) 80 | } 81 | else { 82 | console.log('router next()') 83 | next() 84 | } 85 | }) 86 | 87 | export default router -------------------------------------------------------------------------------- /frontend/src/utils/auth.js: -------------------------------------------------------------------------------- 1 | import decode from 'jwt-decode' 2 | import axios from 'axios' 3 | import * as AmazonCognitoIdentity from 'amazon-cognito-identity-js'; 4 | const AUTH_TOKEN_KEY = 'authToken' 5 | 6 | 7 | export function loginUser(username, password, UserPoolId, ClientId) { 8 | console.log("started login user"); 9 | var authenticationData = { 10 | Username: username, 11 | Password: password 12 | }; 13 | var authenticationDetails = new AmazonCognitoIdentity.AuthenticationDetails(authenticationData); 14 | var poolData = { 15 | UserPoolId: UserPoolId, 16 | ClientId: ClientId 17 | }; 18 | var userPool = new AmazonCognitoIdentity.CognitoUserPool(poolData); 19 | 20 | var userData = { 21 | Username: username, 22 | Pool: userPool 23 | }; 24 | var cognitoUser = new AmazonCognitoIdentity.CognitoUser(userData); 25 | cognitoUser.authenticateUser(authenticationDetails, { 26 | onSuccess: function (result) { 27 | 28 | var accessToken = result.getIdToken().getJwtToken(); 29 | 30 | setAuthToken(accessToken) 31 | console.log(getAuthToken) 32 | if (isLoggedIn()) { 33 | window.location.href = '/'; 34 | } 35 | 36 | }, 37 | onFailure: function (err) { 38 | //isSubmitted =false; 39 | window.alert("Incorrect Username/Password"); 40 | console.log(err) 41 | } 42 | }); 43 | console.log("completed user login"); 44 | 45 | } 46 | 47 | 48 | 49 | 50 | export function logoutUser() { 51 | clearAuthToken() 52 | } 53 | 54 | export function setAuthToken(token) { 55 | console.log('Setting token: ' + token) 56 | axios.defaults.headers.common['Authorization'] = `Bearer ${token}` 57 | localStorage.setItem(AUTH_TOKEN_KEY, token) 58 | let sub = getTokenSub(token) 59 | console.log(`sub: ${sub}`) 60 | } 61 | 62 | export function getAuthToken() { 63 | return localStorage.getItem(AUTH_TOKEN_KEY) 64 | } 65 | 66 | export function clearAuthToken() { 67 | console.log('Clearing auth token') 68 | axios.defaults.headers.common['Authorization'] = '' 69 | localStorage.removeItem(AUTH_TOKEN_KEY) 70 | } 71 | 72 | export function isLoggedIn() { 73 | 74 | let authToken = getAuthToken() 75 | let isLoggedIn = !!authToken && !isTokenExpired(authToken) 76 | console.log(`isLoggedIn: ${isLoggedIn}`) 77 | return isLoggedIn 78 | } 79 | 80 | 81 | export function getUserInfo() { 82 | if (isLoggedIn()) { 83 | return decode(getAuthToken()) 84 | } 85 | } 86 | 87 | export function getUsername() { 88 | if (isLoggedIn()) { 89 | let token = decode(getAuthToken()) 90 | if (!token.username) { 91 | return null 92 | } 93 | 94 | return token.username 95 | } 96 | } 97 | 98 | function getTokenSub(encodedToken) { 99 | let token = decode(encodedToken) 100 | if (!token.sub) { 101 | return null 102 | } 103 | 104 | return token.sub 105 | } 106 | 107 | function getTokenExpirationDate(encodedToken) { 108 | let token = decode(encodedToken) 109 | if (!token.exp) { 110 | return null 111 | } 112 | 113 | let date = new Date(0) 114 | date.setUTCSeconds(token.exp) 115 | 116 | return date 117 | } 118 | 119 | function isTokenExpired(token) { 120 | let expirationDate = getTokenExpirationDate(token) 121 | return expirationDate < new Date() 122 | } 123 | -------------------------------------------------------------------------------- /frontend/src/utils/core.js: -------------------------------------------------------------------------------- 1 | export function sleep(ms) { 2 | return new Promise( 3 | resolve => setTimeout(resolve, ms) 4 | ); 5 | } -------------------------------------------------------------------------------- /frontend/src/views/AboutView.vue: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /frontend/src/views/Home.vue: -------------------------------------------------------------------------------- 1 | 4 | 100 | 160 | -------------------------------------------------------------------------------- /frontend/src/views/KB.vue: -------------------------------------------------------------------------------- 1 | 4 | 77 | 129 | -------------------------------------------------------------------------------- /frontend/src/views/LLMs.vue: -------------------------------------------------------------------------------- 1 | 4 | 111 | 112 | -------------------------------------------------------------------------------- /frontend/src/views/LoginView.vue: -------------------------------------------------------------------------------- 1 | 96 | 97 | 120 | 121 | 149 | 150 | -------------------------------------------------------------------------------- /frontend/src/views/PromptView.vue: -------------------------------------------------------------------------------- 1 | 4 | 102 | 103 | 165 | -------------------------------------------------------------------------------- /frontend/src/views/RAG.vue: -------------------------------------------------------------------------------- 1 | 4 | 104 | 166 | -------------------------------------------------------------------------------- /lambdas/bedrockFunc/cognitouser.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | import secrets 4 | import string 5 | import os 6 | import requests 7 | 8 | REGION = os.environ['AWS_REGION'] 9 | ChatbotUserPool = os.getenv("Cognito_UserPool", None) 10 | ChatbotUserPoolClient = os.getenv("Cognito_ClientID", None) 11 | secrets_name = os.getenv("SECRET_ID", 'ui-credentials') 12 | user_id = os.getenv("USER_ID", 'bedrock') 13 | 14 | 15 | cognitoidentityserviceprovider = boto3.client('cognito-idp', region_name=REGION) 16 | 17 | def generate_random_password(length=12): 18 | # Define character sets 19 | lowercase_letters = string.ascii_lowercase 20 | uppercase_letters = string.ascii_uppercase 21 | digits = string.digits 22 | special_characters = '!$@' 23 | 24 | # Ensure that the password includes at least one special character 25 | special_char = secrets.choice(special_characters) 26 | 27 | # Calculate the remaining length of the password 28 | remaining_length = length - 1 29 | 30 | # Generate the rest of the password with a mix of characters 31 | password_characters = ( 32 | secrets.choice(lowercase_letters) + 33 | secrets.choice(uppercase_letters) + 34 | secrets.choice(digits) + 35 | ''.join(secrets.choice(lowercase_letters + uppercase_letters + digits + special_characters) for _ in range(remaining_length - 3)) 36 | ) 37 | 38 | # Shuffle the characters to make the password random 39 | password_list = list(password_characters) 40 | secrets.SystemRandom().shuffle(password_list) 41 | shuffled_password = ''.join(password_list) 42 | 43 | # Add the special character back at a random position 44 | position = secrets.randbelow(length) 45 | password = shuffled_password[:position] + special_char + shuffled_password[position:] 46 | 47 | return password 48 | 49 | def create_secret(password): 50 | 51 | # Initialize the AWS Secrets Manager client 52 | client = boto3.client('secretsmanager') 53 | 54 | # Store the user ID and password in a dictionary 55 | secret_data = { 56 | 'user_id': user_id, 57 | 'password': password 58 | } 59 | 60 | # Create or update the secret in Secrets Manager 61 | try: 62 | response = client.create_secret( 63 | Name=secrets_name, 64 | SecretString=str(secret_data) 65 | ) 66 | print("Secret created successfully!") 67 | except client.exceptions.ResourceExistsException: 68 | # If the secret already exists, update it 69 | response = client.update_secret( 70 | SecretId=secrets_name, 71 | SecretString=str(secret_data) 72 | ) 73 | print("Secret updated successfully!") 74 | 75 | 76 | def lambda_handler(event, context): 77 | request_type = event['RequestType'] 78 | print('request_type:', request_type) 79 | status_msg = 'SUCCESS' 80 | print('response_url:', event['ResponseURL']) 81 | try: 82 | 83 | if request_type == 'Delete' : 84 | #Delete the secrete 85 | client = boto3.client('secretsmanager') 86 | client.delete_secret(SecretId=secrets_name, ForceDeleteWithoutRecovery=True) 87 | print('workshop secret deleted successful') 88 | status_msg = 'SUCCESS' 89 | else: 90 | password = generate_random_password() 91 | print('password:', password) 92 | create_secret(password) 93 | 94 | try: 95 | cognitoidentityserviceprovider.admin_create_user( 96 | UserPoolId = ChatbotUserPool, 97 | Username = user_id, 98 | UserAttributes = [ 99 | {"Name": "name", "Value": user_id} 100 | ] 101 | ) 102 | except Exception as e1: 103 | print('cognito user exist already') 104 | 105 | cognitoidentityserviceprovider.admin_set_user_password( 106 | UserPoolId = ChatbotUserPool, 107 | Username = user_id, 108 | Password = password, 109 | Permanent=True 110 | ) 111 | status_msg = 'SUCCESS' 112 | except Exception as e: 113 | print('Error: ' + str(e)) 114 | status_msg = 'FAILED' 115 | 116 | 117 | # Respond to CloudFormation to let it know we are done 118 | response_data = build_response(event, status_msg) 119 | response_url = event['ResponseURL'] 120 | requests.put(response_url, data=json.dumps(response_data)) 121 | 122 | print('Cfn signal has been sent back successfully') 123 | 124 | def build_response(event, status): 125 | """A utility function used to build a response to CloudFormation""" 126 | 127 | response_data = { 128 | 'Status': status, 129 | 'Reason': 'Success', 130 | 'PhysicalResourceId': 'myapp::{}'.format(event['LogicalResourceId']), 131 | 'Data': {}, 132 | 'RequestId': event['RequestId'], 133 | "LogicalResourceId": event["LogicalResourceId"], 134 | "StackId": event["StackId"], 135 | } 136 | return response_data 137 | 138 | -------------------------------------------------------------------------------- /lambdas/bedrockFunc/requirements.txt: -------------------------------------------------------------------------------- 1 | boto3>=1.28.57 2 | langchain>=0.0.331 3 | coloredlogs==15.0.1 4 | jq==1.6.0 5 | -------------------------------------------------------------------------------- /lambdas/llmFunctions/kbfunction.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | 4 | 5 | region = boto3.session.Session().region_name 6 | 7 | def lambda_handler(event, context): 8 | print(f"Event is: {event}") 9 | 10 | response = 'This function is not yet available. Please proceed to Task 8, where you will implement it.' 11 | status_code = 200 12 | 13 | 14 | return { 15 | 'statusCode': status_code, 16 | 'headers': { 17 | 'Access-Control-Allow-Origin': '*', 18 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 19 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 20 | }, 21 | 'body': json.dumps({'answer': response}) 22 | } 23 | 24 | -------------------------------------------------------------------------------- /lambdas/llmFunctions/llmfunction.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | import traceback 4 | 5 | 6 | region = boto3.session.Session().region_name 7 | 8 | def lambda_handler(event, context): 9 | print(f"Event is: {event}") 10 | 11 | response = 'This function is not ready, please complete the next two tasks and test again.' 12 | status_code = 200 13 | 14 | 15 | return { 16 | 'statusCode': status_code, 17 | 'headers': { 18 | 'Access-Control-Allow-Origin': '*', 19 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 20 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 21 | }, 22 | 'body': json.dumps({'answer': response}) 23 | } 24 | 25 | -------------------------------------------------------------------------------- /lambdas/ragFunctions/promptfunction.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import boto3 4 | from langchain_community.retrievers import AmazonKendraRetriever 5 | from langchain_aws import ChatBedrock 6 | from langchain.memory import ConversationBufferWindowMemory 7 | from langchain.prompts import PromptTemplate 8 | from langchain.chains import RetrievalQA 9 | import traceback 10 | 11 | # Set up the Kendra client 12 | kendra = boto3.client('kendra') 13 | 14 | KENDRA_INDEX_ID = os.getenv('KENDRA_INDEX_ID') 15 | 16 | def lambda_handler(event, context): 17 | print(f"Event is: {event}") 18 | 19 | event_body = json.loads(event["body"]) 20 | question = event_body["query"] 21 | prompt_template = event_body["prompt"] 22 | print(f"Query: {question}") 23 | print(f"Prompt: {prompt_template}") 24 | 25 | model_id = event_body["model_id"] 26 | temperature = event_body["temperature"] 27 | max_tokens = event_body["max_tokens"] 28 | 29 | response = '' 30 | status_code = 200 31 | 32 | try: 33 | llm = get_claude_llm(model_id, temperature, max_tokens) 34 | 35 | # Initialize the Kendra loader 36 | retriever = AmazonKendraRetriever( 37 | kendra_client=kendra, 38 | index_id=KENDRA_INDEX_ID 39 | ) 40 | claude_prompt = PromptTemplate( 41 | template=prompt_template, input_variables=["context","question"] 42 | ) 43 | 44 | qa = RetrievalQA.from_chain_type( 45 | llm=llm, 46 | chain_type="stuff", 47 | retriever=retriever, 48 | return_source_documents=False, 49 | chain_type_kwargs={"prompt": claude_prompt} 50 | ) 51 | 52 | response = qa(question, return_only_outputs=True) 53 | print(response) 54 | 55 | response_with_metadata = { 56 | "answer": response['result'] 57 | } 58 | 59 | 60 | return { 61 | 'statusCode': status_code, 62 | 'headers': { 63 | 'Access-Control-Allow-Origin': '*', 64 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 65 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 66 | }, 67 | 'body': json.dumps(response_with_metadata) 68 | } 69 | 70 | except Exception as e: 71 | print(f"An unexpected error occurred: {str(e)}") 72 | stack_trace = traceback.format_exc() 73 | print(f"stack trace: {stack_trace}") 74 | print(f"error: {str(e)}") 75 | 76 | response = str(e) 77 | return { 78 | 'statusCode': status_code, 79 | 'headers': { 80 | 'Access-Control-Allow-Origin': '*', 81 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 82 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 83 | }, 84 | 'body': json.dumps({'error': response}) 85 | } 86 | 87 | def get_claude_llm(model_id, temperature, max_tokens): 88 | model_kwargs = { 89 | "max_tokens": max_tokens, 90 | "temperature": temperature, 91 | "top_k": 50, 92 | "top_p": 1 93 | } 94 | llm = ChatBedrock(model_id=model_id, model_kwargs=model_kwargs) 95 | return llm 96 | 97 | #This is a TODO item, presently the history is not retained between the calls 98 | def get_memory(): 99 | memory = ConversationBufferWindowMemory( 100 | memory_key="chat_history", 101 | k=5, 102 | input_key="question", 103 | output_key="answer", 104 | return_messages=True) 105 | return memory 106 | -------------------------------------------------------------------------------- /lambdas/ragFunctions/ragfunction.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | import json 4 | import traceback 5 | 6 | 7 | region = boto3.session.Session().region_name 8 | 9 | def lambda_handler(event, context): 10 | print(f"Event is: {event}") 11 | 12 | response = 'This function is not ready, please complete the next two tasks and test again.' 13 | status_code = 200 14 | 15 | 16 | return { 17 | 'statusCode': status_code, 18 | 'headers': { 19 | 'Access-Control-Allow-Origin': '*', 20 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 21 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 22 | }, 23 | 'body': json.dumps({'answer': response}) 24 | } 25 | 26 | -------------------------------------------------------------------------------- /lambdas/ragFunctions/requirements.txt: -------------------------------------------------------------------------------- 1 | #This project will require modification to support langchain 0.3.0, till then use this version 0.2.16 2 | langchain==0.2.16 3 | langchain-community 4 | langchain-aws 5 | -------------------------------------------------------------------------------- /prompt-engineering/claude-prompt-template.txt: -------------------------------------------------------------------------------- 1 | Human: You are an AI assistant specialized Healthcare and Life Sciences area, and providing information and advices in that area. We expect you to provide answers to questions using fact-based information available from the context only. If the provided context is insufficient to answer the question, politely indicate that and refrain from speculating or making up information. 2 | 3 | The response should be specific, use facts only, and when possible. When providing factual information, cite the relevant sources from the context. 4 | 5 | If the question is unclear or ambiguous, feel free to ask for clarification before providing an answer. 6 | 7 | If the context contains sensitive or confidential information, refrain from disclosing or discussing it in your response. 8 | 9 | If you don't know the answer, just say that you don't know, don't try to make up an answer. 10 | 11 | Use the following information to provide a concise answer to user query in tags. Look for the contextual information enclosed in tags. 12 | 13 | {context} 14 | 15 | {question} 16 | 17 | Assistant: -------------------------------------------------------------------------------- /prompt-engineering/llama-prompt-template.txt: -------------------------------------------------------------------------------- 1 | [INST] 2 | You are an AI assistant specialized Healthcare and Life Sciences area, and providing information and advices in that area. We expect you to provide answers to questions using fact-based information available from the context only. If the provided context is insufficient to answer the question, politely indicate that and refrain from speculating or making up information. 3 | 4 | The response should be specific, use facts only, and when possible. When providing factual information, cite the relevant sources from the context. 5 | 6 | If the question is unclear or ambiguous, feel free to ask for clarification before providing an answer. 7 | 8 | If the context contains sensitive or confidential information, refrain from disclosing or discussing it in your response. 9 | 10 | If you don't know the answer, just say that you don't know, don't try to make up an answer. 11 | 12 | Use the following information to provide a concise answer to user query in tags. Look for the contextual information enclosed in tags. 13 | You do not have to include the original question in your response. 14 | 15 | {context} 16 | 17 | {question} 18 | 19 | [/INST] 20 | 21 | -------------------------------------------------------------------------------- /prompt-engineering/mistral-prompt-template.txt: -------------------------------------------------------------------------------- 1 | [INST] 2 | You are an AI assistant specialized Healthcare and Life Sciences area, and providing information and advices in that area. We expect you to provide answers to questions using fact-based information available from the context only. If the provided context is insufficient to answer the question, politely indicate that and refrain from speculating or making up information. 3 | 4 | The response should be specific, use facts only, and when possible. When providing factual information, cite the relevant sources from the context. 5 | 6 | If the question is unclear or ambiguous, feel free to ask for clarification before providing an answer. 7 | 8 | If the context contains sensitive or confidential information, refrain from disclosing or discussing it in your response. 9 | 10 | If you don't know the answer, just say that you don't know, don't try to make up an answer. 11 | 12 | Use the following information to provide a concise answer to user query in tags. Look for the contextual information enclosed in tags. 13 | 14 | {context} 15 | 16 | {question} 17 | 18 | [/INST] 19 | 20 | -------------------------------------------------------------------------------- /startup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | echo "Preparing for project deploymnet" 3 | # Check if 'CFNStackName' is set in the environment variables 4 | if [ -z "$CFNStackName" ]; then 5 | echo "Error: 'CFNStackName' environment variable is not set. Please set it and run the script." 6 | exit 1 7 | fi 8 | echo "CFN Start up stack name: $CFNStackName" 9 | 10 | 11 | echo "Set region" 12 | export AWS_REGION=$(aws configure get region) 13 | echo "AWS Region is $AWS_REGION" 14 | 15 | echo "Build the backend code using sam build" 16 | cd ~/environment/bedrock-serverless-workshop 17 | sam build 18 | 19 | echo "Export S3 bucket name and Kendra index which are created as part of Startup CFN stack" 20 | export S3BucketName=$(aws cloudformation describe-stacks --stack-name ${CFNStackName} --query "Stacks[0].Outputs[?OutputKey=='S3BucketName'].OutputValue" --output text) 21 | export KendraIndexID=$(aws cloudformation describe-stacks --stack-name ${CFNStackName} --query "Stacks[0].Outputs[?OutputKey=='KendraIndexID'].OutputValue" --output text) 22 | 23 | export SAMStackName="sam-$CFNStackName" 24 | echo $SAMStackName 25 | 26 | echo "Copy toml file and replace the parameters" 27 | 28 | cp tools/samconf.toml samconfig.toml 29 | # Replace values in .//samconfig.toml 30 | sed -Ei "s||${KendraIndexID}|g" ./samconfig.toml 31 | sed -Ei "s||${S3BucketName}|g" ./samconfig.toml 32 | sed -Ei "s||${SAMStackName}|g" ./samconfig.toml 33 | sed -Ei "s||${AWS_REGION}|g" ./samconfig.toml 34 | 35 | echo "Deploy app with sam deploy" 36 | sam deploy 37 | 38 | echo "Export few more parameters from the sam stack output" 39 | export BedrockApiUrl=$(aws cloudformation describe-stacks --stack-name ${SAMStackName} --query "Stacks[0].Outputs[?OutputKey=='BedrockApiUrl'].OutputValue" --output text) 40 | export UserPoolId=$(aws cloudformation describe-stacks --stack-name ${SAMStackName} --query "Stacks[0].Outputs[?OutputKey=='CognitoUserPool'].OutputValue" --output text) 41 | export UserPoolClientId=$(aws cloudformation describe-stacks --stack-name ${SAMStackName} --query "Stacks[0].Outputs[?OutputKey=='CongnitoUserPoolClientID'].OutputValue" --output text) 42 | export SecretName=$(aws cloudformation describe-stacks --stack-name ${SAMStackName} --query "Stacks[0].Outputs[?OutputKey=='SecretsName'].OutputValue" --output text) 43 | echo "API Gateway endpoint: $BedrockApiUrl" 44 | echo "Cognito user pool id: $UserPoolId" 45 | echo "Cognito client id: $UserPoolClientId" 46 | echo "Secret name: $SecretName" 47 | 48 | # Replace values in ./backend/samconfig.toml 49 | sed -Ei "s||${BedrockApiUrl}|g" ./frontend/src/main.js 50 | sed -Ei "s||${UserPoolId}|g" ./frontend/src/main.js 51 | sed -Ei "s||${UserPoolClientId}|g" ./frontend/src/main.js 52 | 53 | #Pre req for frontend setup 54 | mkdir ~/.npm-global 55 | npm config set prefix '~/.npm-global' 56 | export PATH=~/.npm-global/bin:$PATH 57 | 58 | #Install Ampliyfy and build frontend 59 | #this export to fix the npm installation issue 60 | export UV_USE_IO_URING=0 61 | cd /home/ec2-user/environment/bedrock-serverless-workshop/frontend 62 | npm i -S @vue/cli-service 63 | npm i @vue/cli-plugin-babel -D 64 | npm i -g @aws-amplify/cli 65 | npm install 66 | npm run build 67 | cp ~/.aws/credentials ~/.aws/config 68 | 69 | #Amplify initialization 70 | echo "Amplify initialization" 71 | mv dist build 72 | amplify init --yes 73 | 74 | echo "Add hosting, hit enter key if it prompts for action, use default" 75 | amplify add hosting --parameters parameters.json 76 | 77 | echo "Publish the amplify project" 78 | amplify publish --yes 79 | 80 | echo "Save the user_id and password to login to UI" 81 | aws secretsmanager get-secret-value --secret-id $SecretName | jq -r .SecretString 82 | -------------------------------------------------------------------------------- /template.yaml: -------------------------------------------------------------------------------- 1 | AWSTemplateFormatVersion: '2010-09-09' 2 | Transform: AWS::Serverless-2016-10-31 3 | Description: 4 | SAM template for serverless bedrock chatbot workshop. 5 | 6 | Parameters: 7 | KendraIndexId: 8 | Description: ID of the Kendra Index 9 | Type: String 10 | 11 | BedrockWSS3Bucket: 12 | Description: Workshop bucket name 13 | Type: String 14 | 15 | 16 | ApiGatewayStageName: 17 | Default: prod 18 | Description : Stage name for the API Gateway 19 | Type: String 20 | 21 | # More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst 22 | Globals: 23 | Function: 24 | Timeout: 60 25 | MemorySize: 3000 26 | Tracing: Active 27 | Api: 28 | TracingEnabled: true 29 | Cors: 30 | AllowMethods: "'GET,POST,OPTIONS'" 31 | AllowHeaders: "'Content-Type', 'Authorization', 'X-Forwarded-For', 'X-Api-Key', 'X-Amz-Date', 'X-Amz-Security-Token'" 32 | AllowOrigin: "'*'" 33 | 34 | Resources: 35 | # REST API 36 | BedrockLambdaApi: 37 | Type: AWS::ApiGateway::RestApi 38 | Properties: 39 | Name: bedrock-workshop 40 | Description: Mock Integration REST API demo 41 | 42 | 43 | 44 | RagResource: 45 | Type: AWS::ApiGateway::Resource 46 | Properties: 47 | ParentId: !GetAtt BedrockLambdaApi.RootResourceId 48 | PathPart: 'rag' 49 | RestApiId: !Ref BedrockLambdaApi 50 | 51 | LLMResource: 52 | Type: AWS::ApiGateway::Resource 53 | Properties: 54 | ParentId: !GetAtt BedrockLambdaApi.RootResourceId 55 | PathPart: 'llms' 56 | RestApiId: !Ref BedrockLambdaApi 57 | 58 | PromptResource: 59 | Type: AWS::ApiGateway::Resource 60 | Properties: 61 | ParentId: !GetAtt BedrockLambdaApi.RootResourceId 62 | PathPart: 'prompt' 63 | RestApiId: !Ref BedrockLambdaApi 64 | 65 | KBResource: 66 | Type: AWS::ApiGateway::Resource 67 | Properties: 68 | ParentId: !GetAtt BedrockLambdaApi.RootResourceId 69 | PathPart: 'kb' 70 | RestApiId: !Ref BedrockLambdaApi 71 | 72 | # KB Method Config 73 | KBOptionsMethod: 74 | Type: AWS::ApiGateway::Method 75 | Properties: 76 | HttpMethod: OPTIONS 77 | ResourceId: !Ref KBResource 78 | RestApiId: !Ref BedrockLambdaApi 79 | AuthorizationType: NONE 80 | Integration: 81 | IntegrationResponses: 82 | - StatusCode: 200 83 | ResponseParameters: 84 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 85 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 86 | method.response.header.Access-Control-Allow-Origin: "'*'" 87 | PassthroughBehavior: WHEN_NO_MATCH 88 | RequestTemplates: 89 | application/json: '{"statusCode": 200}' 90 | Type: MOCK 91 | MethodResponses: 92 | - StatusCode: 200 93 | ResponseModels: 94 | application/json: 'Empty' 95 | ResponseParameters: 96 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 97 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 98 | method.response.header.Access-Control-Allow-Origin: "'*'" 99 | 100 | KBPostMethod: 101 | Type: AWS::ApiGateway::Method 102 | Properties: 103 | HttpMethod: POST 104 | ResourceId: !Ref KBResource 105 | RestApiId: !Ref BedrockLambdaApi 106 | AuthorizationType: NONE 107 | Integration: 108 | IntegrationHttpMethod: POST 109 | Type: AWS_PROXY 110 | Uri: !Sub "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${KBFunction.Arn}/invocations" 111 | MethodResponses: 112 | - StatusCode: 200 113 | ResponseParameters: 114 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 115 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 116 | method.response.header.Access-Control-Allow-Origin: "'*'" 117 | 118 | KBFunctionPermission: 119 | Type: AWS::Lambda::Permission 120 | Properties: 121 | FunctionName: !Ref KBFunction 122 | Action: lambda:InvokeFunction 123 | Principal: apigateway.amazonaws.com 124 | SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${BedrockLambdaApi}/*/*/*" 125 | 126 | KBApiDeployment: 127 | Type: AWS::ApiGateway::Deployment 128 | DependsOn: KBPostMethod 129 | Properties: 130 | Description: KB API Deployment 131 | RestApiId: !Ref BedrockLambdaApi 132 | 133 | # RAG Method Config 134 | RagOptionsMethod: 135 | Type: AWS::ApiGateway::Method 136 | Properties: 137 | HttpMethod: OPTIONS 138 | ResourceId: !Ref RagResource 139 | RestApiId: !Ref BedrockLambdaApi 140 | AuthorizationType: NONE 141 | Integration: 142 | IntegrationResponses: 143 | - StatusCode: 200 144 | ResponseParameters: 145 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 146 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 147 | method.response.header.Access-Control-Allow-Origin: "'*'" 148 | PassthroughBehavior: WHEN_NO_MATCH 149 | RequestTemplates: 150 | application/json: '{"statusCode": 200}' 151 | Type: MOCK 152 | MethodResponses: 153 | - StatusCode: 200 154 | ResponseModels: 155 | application/json: 'Empty' 156 | ResponseParameters: 157 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 158 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 159 | method.response.header.Access-Control-Allow-Origin: "'*'" 160 | 161 | RagPostMethod: 162 | Type: AWS::ApiGateway::Method 163 | Properties: 164 | HttpMethod: POST 165 | ResourceId: !Ref RagResource 166 | RestApiId: !Ref BedrockLambdaApi 167 | AuthorizationType: NONE 168 | Integration: 169 | IntegrationHttpMethod: POST 170 | Type: AWS_PROXY 171 | Uri: !Sub "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${RagFunction.Arn}/invocations" 172 | MethodResponses: 173 | - StatusCode: 200 174 | ResponseParameters: 175 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 176 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 177 | method.response.header.Access-Control-Allow-Origin: "'*'" 178 | 179 | RagFunctionPermission: 180 | Type: AWS::Lambda::Permission 181 | Properties: 182 | FunctionName: !Ref RagFunction 183 | Action: lambda:InvokeFunction 184 | Principal: apigateway.amazonaws.com 185 | SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${BedrockLambdaApi}/*/*/*" 186 | 187 | RagApiDeployment: 188 | Type: AWS::ApiGateway::Deployment 189 | DependsOn: RagPostMethod 190 | Properties: 191 | Description: RAG API Deployment 192 | RestApiId: !Ref BedrockLambdaApi 193 | 194 | # Prompt Method Config 195 | PromptOptionsMethod: 196 | Type: AWS::ApiGateway::Method 197 | Properties: 198 | HttpMethod: OPTIONS 199 | ResourceId: !Ref PromptResource 200 | RestApiId: !Ref BedrockLambdaApi 201 | AuthorizationType: NONE 202 | Integration: 203 | IntegrationResponses: 204 | - StatusCode: 200 205 | ResponseParameters: 206 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 207 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 208 | method.response.header.Access-Control-Allow-Origin: "'*'" 209 | PassthroughBehavior: WHEN_NO_MATCH 210 | RequestTemplates: 211 | application/json: '{"statusCode": 200}' 212 | Type: MOCK 213 | MethodResponses: 214 | - StatusCode: 200 215 | ResponseModels: 216 | application/json: 'Empty' 217 | ResponseParameters: 218 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 219 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 220 | method.response.header.Access-Control-Allow-Origin: "'*'" 221 | 222 | PromptPostMethod: 223 | Type: AWS::ApiGateway::Method 224 | Properties: 225 | HttpMethod: POST 226 | ResourceId: !Ref PromptResource 227 | RestApiId: !Ref BedrockLambdaApi 228 | AuthorizationType: NONE 229 | Integration: 230 | IntegrationHttpMethod: POST 231 | Type: AWS_PROXY 232 | Uri: !Sub "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${PromptFunction.Arn}/invocations" 233 | MethodResponses: 234 | - StatusCode: 200 235 | ResponseParameters: 236 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 237 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 238 | method.response.header.Access-Control-Allow-Origin: "'*'" 239 | 240 | PromptFunctionPermission: 241 | Type: AWS::Lambda::Permission 242 | Properties: 243 | FunctionName: !Ref PromptFunction 244 | Action: lambda:InvokeFunction 245 | Principal: apigateway.amazonaws.com 246 | SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${BedrockLambdaApi}/*/*/*" 247 | 248 | PromptApiDeployment: 249 | Type: AWS::ApiGateway::Deployment 250 | DependsOn: PromptPostMethod 251 | Properties: 252 | Description: RAG API Deployment 253 | RestApiId: !Ref BedrockLambdaApi 254 | 255 | # LLM Method Config 256 | LLMOptionsMethod: 257 | Type: AWS::ApiGateway::Method 258 | Properties: 259 | HttpMethod: OPTIONS 260 | ResourceId: !Ref LLMResource 261 | RestApiId: !Ref BedrockLambdaApi 262 | AuthorizationType: NONE 263 | Integration: 264 | IntegrationResponses: 265 | - StatusCode: 200 266 | ResponseParameters: 267 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 268 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 269 | method.response.header.Access-Control-Allow-Origin: "'*'" 270 | PassthroughBehavior: WHEN_NO_MATCH 271 | RequestTemplates: 272 | application/json: '{"statusCode": 200}' 273 | Type: MOCK 274 | MethodResponses: 275 | - StatusCode: 200 276 | ResponseModels: 277 | application/json: 'Empty' 278 | ResponseParameters: 279 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 280 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 281 | method.response.header.Access-Control-Allow-Origin: "'*'" 282 | 283 | LLMPostMethod: 284 | Type: AWS::ApiGateway::Method 285 | Properties: 286 | HttpMethod: POST 287 | ResourceId: !Ref LLMResource 288 | RestApiId: !Ref BedrockLambdaApi 289 | AuthorizationType: NONE 290 | Integration: 291 | IntegrationHttpMethod: POST 292 | Type: AWS_PROXY 293 | Uri: !Sub "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LLMFunction.Arn}/invocations" 294 | MethodResponses: 295 | - StatusCode: 200 296 | ResponseParameters: 297 | method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" 298 | method.response.header.Access-Control-Allow-Methods: "'POST,OPTIONS'" 299 | method.response.header.Access-Control-Allow-Origin: "'*'" 300 | 301 | LLMFunctionPermission: 302 | Type: AWS::Lambda::Permission 303 | Properties: 304 | FunctionName: !Ref LLMFunction 305 | Action: lambda:InvokeFunction 306 | Principal: apigateway.amazonaws.com 307 | SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${BedrockLambdaApi}/*/*/*" 308 | 309 | LLMApiDeployment: 310 | Type: AWS::ApiGateway::Deployment 311 | DependsOn: LLMPostMethod 312 | Properties: 313 | Description: LLM API Deployment 314 | RestApiId: !Ref BedrockLambdaApi 315 | 316 | #### Cognito Auth Config 317 | ApiCognitoAuthorizer: 318 | Type: AWS::ApiGateway::Authorizer 319 | Properties: 320 | IdentitySource: 'method.request.header.Authorization' 321 | Name: ApiCognitoAuthorizer 322 | ProviderARNs: 323 | - !GetAtt ChatbotUserPool.Arn 324 | RestApiId: !Ref BedrockLambdaApi 325 | Type: COGNITO_USER_POOLS 326 | 327 | ChatbotUserPool: 328 | Type: AWS::Cognito::UserPool 329 | Properties: 330 | UsernameConfiguration: 331 | CaseSensitive: false 332 | AutoVerifiedAttributes: 333 | - email 334 | Schema: 335 | - Name: email 336 | AttributeDataType: String 337 | Mutable: false 338 | Required: false 339 | - Name: name 340 | AttributeDataType: String 341 | Mutable: true 342 | Required: true 343 | ChatbotUserPoolClient: 344 | Type: AWS::Cognito::UserPoolClient 345 | Properties: 346 | UserPoolId: !Ref ChatbotUserPool 347 | ExplicitAuthFlows: 348 | - ALLOW_USER_PASSWORD_AUTH 349 | - ALLOW_REFRESH_TOKEN_AUTH 350 | - ALLOW_USER_SRP_AUTH 351 | - ALLOW_CUSTOM_AUTH 352 | AllowedOAuthFlowsUserPoolClient: true 353 | CallbackURLs: 354 | - http://localhost:3000 355 | AllowedOAuthFlows: 356 | - code 357 | - implicit 358 | AllowedOAuthScopes: 359 | - phone 360 | - email 361 | - openid 362 | - profile 363 | SupportedIdentityProviders: 364 | - COGNITO 365 | 366 | CognitoUserCreateFunction: 367 | Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction 368 | Properties: 369 | CodeUri: lambdas/bedrockFunc 370 | Handler: cognitouser.lambda_handler 371 | Runtime: python3.11 372 | MemorySize: 2048 373 | Architectures: 374 | - x86_64 375 | Environment: 376 | Variables: 377 | Cognito_UserPool: !Ref ChatbotUserPool 378 | Cognito_ClientID: !Ref ChatbotUserPoolClient 379 | SECRET_ID: 380 | Fn::Sub: "ui-credentials-${BedrockLambdaApi}" 381 | Policies: 382 | - Version: '2012-10-17' 383 | Statement: 384 | - Effect: Allow 385 | Action: 386 | - secretsmanager:Get* 387 | - secretsmanager:Describe* 388 | - secretsmanager:CreateSecret 389 | - secretsmanager:DeleteSecret 390 | - secretsmanager:UpdateSecret 391 | Resource: 392 | - !Sub "arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:*" 393 | - Effect: Allow 394 | Action: 395 | - cognito-idp:Describe* 396 | - cognito-idp:CreateUserPool 397 | - cognito-idp:CreateUserPoolClient 398 | - cognito-idp:DeleteUserPool 399 | - cognito-idp:UpdateUserPool 400 | - cognito-idp:Admin* 401 | Resource: 402 | - !Sub "arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/*" 403 | - !Sub "arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/*/*" 404 | 405 | DeploymentCustomResource: 406 | Type: Custom::AppConfiguration 407 | Properties: 408 | ServiceToken: !GetAtt CognitoUserCreateFunction.Arn 409 | 410 | ApiGatewayStage: 411 | Type: AWS::ApiGateway::Stage 412 | Properties: 413 | DeploymentId: !Ref LLMApiDeployment 414 | Description: Lambda API Stage 415 | RestApiId: !Ref BedrockLambdaApi 416 | StageName: !Ref ApiGatewayStageName 417 | 418 | 419 | LLMFunction: 420 | Type: AWS::Serverless::Function 421 | Properties: 422 | Handler: llmfunction.lambda_handler 423 | CodeUri: lambdas/llmFunctions 424 | Runtime: python3.11 425 | Architectures: 426 | - x86_64 427 | Role: !GetAtt LLMExecutionRole.Arn 428 | 429 | RagFunction: 430 | Type: AWS::Serverless::Function 431 | Properties: 432 | Handler: ragfunction.lambda_handler 433 | CodeUri: lambdas/ragFunctions 434 | Runtime: python3.11 435 | Architectures: 436 | - x86_64 437 | Environment: 438 | Variables: 439 | KENDRA_INDEX_ID: !Ref KendraIndexId 440 | S3_BUCKET_NAME: !Ref BedrockWSS3Bucket 441 | Role: !GetAtt LLMExecutionRole.Arn 442 | 443 | PromptFunction: 444 | Type: AWS::Serverless::Function 445 | Properties: 446 | Handler: promptfunction.lambda_handler 447 | CodeUri: lambdas/ragFunctions 448 | Runtime: python3.11 449 | Architectures: 450 | - x86_64 451 | Environment: 452 | Variables: 453 | KENDRA_INDEX_ID: !Ref KendraIndexId 454 | S3_BUCKET_NAME: !Ref BedrockWSS3Bucket 455 | Role: !GetAtt LLMExecutionRole.Arn 456 | 457 | KBFunction: 458 | Type: AWS::Serverless::Function 459 | Properties: 460 | Handler: kbfunction.lambda_handler 461 | CodeUri: lambdas/llmFunctions 462 | Runtime: python3.11 463 | Architectures: 464 | - x86_64 465 | Environment: 466 | Variables: 467 | KB_ID: copy_kb_id 468 | Role: !GetAtt LLMExecutionRole.Arn 469 | 470 | LLMExecutionRole: 471 | Type: AWS::IAM::Role 472 | Properties: 473 | AssumeRolePolicyDocument: 474 | Version: '2012-10-17' 475 | Statement: 476 | - Effect: Allow 477 | Principal: 478 | Service: lambda.amazonaws.com 479 | Action: sts:AssumeRole 480 | Path: / 481 | 482 | LLMExecutionPolicy: 483 | Type: AWS::IAM::Policy 484 | Properties: 485 | PolicyName: LLMExecutionPolicy 486 | Roles: 487 | - !Ref LLMExecutionRole 488 | PolicyDocument: 489 | Version: '2012-10-17' 490 | Statement: 491 | - Effect: Allow 492 | Action: 493 | - kendra:Query 494 | - kendra:GetQuerySuggestions 495 | - kendra:Retrieve 496 | - s3:GetObject 497 | - s3:ListBucket 498 | - s3:ListBucketVersions 499 | - s3:DescribeBucket 500 | - bedrock:ListFoundationModels 501 | - bedrock:GetFoundationModel 502 | - bedrock:InvokeModel 503 | - bedrock:InvokeModelWithResponseStream 504 | - bedrock:ListTagsForResource 505 | - bedrock:UntagResource 506 | - bedrock:TagResource 507 | - bedrock:AcceptAcknowledgement 508 | - bedrock:GetModelPermission 509 | - bedrock:GetModelInvocationLogging 510 | - bedrock:PutModelInvocationLogging 511 | - bedrock:GetKnowledgeBase 512 | - bedrock:RetrieveAndGenerate 513 | - bedrock:Retrieve 514 | - logs:CreateLogGroup 515 | - logs:CreateLogStream 516 | - logs:PutLogEvents 517 | Resource: 518 | - !Sub "arn:aws:kendra:${AWS::Region}:${AWS::AccountId}:index/${KendraIndexId}" 519 | - !Sub "arn:aws:s3:::${BedrockWSS3Bucket}/*" 520 | - !Sub "arn:aws:s3:::${BedrockWSS3Bucket}" 521 | - "*" 522 | BedrockExecutionRoleForKBs: 523 | Type: 'AWS::IAM::Role' 524 | Properties: 525 | AssumeRolePolicyDocument: 526 | Version: 2012-10-17 527 | Statement: 528 | - Sid: '' 529 | Effect: Allow 530 | Principal: 531 | Service: bedrock.amazonaws.com 532 | Action: 'sts:AssumeRole' 533 | Policies: 534 | - PolicyName: BedrockOpenSearchS3AccessPolicy 535 | PolicyDocument: 536 | Version: '2012-10-17' 537 | Statement: 538 | - Effect: Allow 539 | Action: 540 | - 'bedrock:InvokeModel' 541 | Resource: 542 | - !Sub "arn:aws:bedrock:${AWS::Region}::foundation-model/amazon.titan-embed-text-v2:0" 543 | - Effect: Allow 544 | Action: 545 | - 'aoss:APIAccessAll' 546 | Resource: 547 | - !Sub "arn:aws:aoss:${AWS::Region}:${AWS::AccountId}:collection/*" 548 | - Effect: Allow 549 | Action: 550 | - 's3:ListBucket' 551 | - 's3:GetObject' 552 | Resource: 553 | - !Sub "arn:aws:s3:::${BedrockWSS3Bucket}/sample-documents/*" 554 | - !Sub "arn:aws:s3:::${BedrockWSS3Bucket}" 555 | 556 | Outputs: 557 | CognitoUserPool: 558 | Description: Cognito User Pool 559 | Value: 560 | Fn::Sub: "${ChatbotUserPool}" 561 | 562 | CongnitoUserPoolClientID: 563 | Description: Cognito User Pool Client ID 564 | Value: 565 | Fn::Sub: "${ChatbotUserPoolClient}" 566 | 567 | BedrockApiUrl: 568 | Description: API Gateway endpoint URL for the Bedrock Lambda Function 569 | Value: 570 | Fn::Sub: "https://${BedrockLambdaApi}.execute-api.${AWS::Region}.${AWS::URLSuffix}/prod" 571 | 572 | SecretsName: 573 | Description: Secrets name to retrieve ui credentials 574 | Value: 575 | Fn::Sub: "ui-credentials-${BedrockLambdaApi}" 576 | -------------------------------------------------------------------------------- /tools/account-setup.sh: -------------------------------------------------------------------------------- 1 | # Create AWS accounts 2 | for i in {18..20}; do 3 | aws organizations create-account --email prallam+acnt$i@amazon.com --account-name aws-events-acnt$i 4 | done 5 | 6 | # Move the accounts into the 'bedrockevent' organizational unit 7 | OU_ID=$(aws organizations list-organizational-units | jq -r '.OrganizationalUnits[] | select(.Name == "bedrockevent").Id') 8 | for i in {18..20}; do 9 | ACCOUNT_ID=$(aws organizations list-accounts | jq -r '.Accounts[] | select(.Name == "aws-events-acnt'$i'").Id') 10 | aws organizations move-account --account-id $ACCOUNT_ID --source-parent-id ou-7439 --destination-parent-id $OU_ID 11 | done 12 | 13 | aws organizations move-account --account-id 525840005564 --source-parent-id ou-7439 --destination-parent-id ou-7439-16nv0zlq 14 | 15 | 16 | #Create 5 IAM users CFN call 17 | #aws cloudformation create-stack --stack-name user-setup-DONOT-DELETE --template-body file://gen-ai-builder-init-v1.2.yaml --capabilities CAPABILITY_NAMED_IAM 18 | -------------------------------------------------------------------------------- /tools/gen-ai-builder-init v1.1.yml: -------------------------------------------------------------------------------- 1 | Metadata: 2 | AWS::CloudFormation::Interface: 3 | ParameterGroups: [] 4 | ParameterLabels: {} 5 | Comments: Prepares Account User Access for GenAI Builder Event 6 | CreatedBy: Carter Meyers (AWS) 7 | Description: GenAI Builder Event Prep - v1.0 8 | LastUpdated: October 18, 2023 9 | Version: v1.0 10 | Outputs: 11 | User01: 12 | Value: !Join 13 | - '' 14 | - - 'Sign-in URL: https://' 15 | - !Ref 'AWS::AccountId' 16 | - '.signin.aws.amazon.com/console | Username: ' 17 | - !Ref 'User01' 18 | - ' | Password: ' 19 | - GenAIBuilder! 20 | User02: 21 | Value: !Join 22 | - '' 23 | - - 'Sign-in URL: https://' 24 | - !Ref 'AWS::AccountId' 25 | - '.signin.aws.amazon.com/console | Username: ' 26 | - !Ref 'User02' 27 | - ' | Password: ' 28 | - GenAIBuilder! 29 | User03: 30 | Value: !Join 31 | - '' 32 | - - 'Sign-in URL: https://' 33 | - !Ref 'AWS::AccountId' 34 | - '.signin.aws.amazon.com/console | Username: ' 35 | - !Ref 'User03' 36 | - ' | Password: ' 37 | - GenAIBuilder! 38 | User04: 39 | Value: !Join 40 | - '' 41 | - - 'Sign-in URL: https://' 42 | - !Ref 'AWS::AccountId' 43 | - '.signin.aws.amazon.com/console | Username: ' 44 | - !Ref 'User04' 45 | - ' | Password: ' 46 | - GenAIBuilder! 47 | User05: 48 | Value: !Join 49 | - '' 50 | - - 'Sign-in URL: https://' 51 | - !Ref 'AWS::AccountId' 52 | - '.signin.aws.amazon.com/console | Username: ' 53 | - !Ref 'User05' 54 | - ' | Password: ' 55 | - GenAIBuilder! 56 | User06: 57 | Value: !Join 58 | - '' 59 | - - 'Sign-in URL: https://' 60 | - !Ref 'AWS::AccountId' 61 | - '.signin.aws.amazon.com/console | Username: ' 62 | - !Ref 'User06' 63 | - ' | Password: ' 64 | - GenAIBuilder! 65 | User07: 66 | Value: !Join 67 | - '' 68 | - - 'Sign-in URL: https://' 69 | - !Ref 'AWS::AccountId' 70 | - '.signin.aws.amazon.com/console | Username: ' 71 | - !Ref 'User07' 72 | - ' | Password: ' 73 | - GenAIBuilder! 74 | User08: 75 | Value: !Join 76 | - '' 77 | - - 'Sign-in URL: https://' 78 | - !Ref 'AWS::AccountId' 79 | - '.signin.aws.amazon.com/console | Username: ' 80 | - !Ref 'User08' 81 | - ' | Password: ' 82 | - GenAIBuilder! 83 | User09: 84 | Value: !Join 85 | - '' 86 | - - 'Sign-in URL: https://' 87 | - !Ref 'AWS::AccountId' 88 | - '.signin.aws.amazon.com/console | Username: ' 89 | - !Ref 'User09' 90 | - ' | Password: ' 91 | - GenAIBuilder! 92 | User10: 93 | Value: !Join 94 | - '' 95 | - - 'Sign-in URL: https://' 96 | - !Ref 'AWS::AccountId' 97 | - '.signin.aws.amazon.com/console | Username: ' 98 | - !Ref 'User10' 99 | - ' | Password: ' 100 | - GenAIBuilder! 101 | Resources: 102 | IAMGroup: 103 | Properties: 104 | ManagedPolicyArns: [] 105 | Policies: 106 | - PolicyDocument: 107 | Statement: 108 | - Action: 109 | - sns:* 110 | - cognito-identity:* 111 | - s3:* 112 | - apigateway:* 113 | - cloudformation:* 114 | - cloud9:* 115 | - amplify:* 116 | - iam:* 117 | - kendra:* 118 | - serverlessrepo:* 119 | - lambda:* 120 | - logs:* 121 | - cloudwatch:* 122 | - ec2:* 123 | - cognito-idp:* 124 | - secretsmanager:* 125 | - aws-marketplace:* 126 | - bedrock:* 127 | Effect: Allow 128 | Resource: 129 | - '*' 130 | - Action: 131 | - bedrock:UpdateProvisionedModelThroughput 132 | - bedrock:DeleteProvisionedModelThroughput 133 | - bedrock:CreateProvisionedModelThroughput 134 | Effect: Deny 135 | Resource: 136 | - '*' 137 | PolicyName: gen-ai-builder-access 138 | Type: AWS::IAM::Group 139 | 140 | User01: 141 | Properties: 142 | UserName: User01 143 | Groups: 144 | - !Ref 'IAMGroup' 145 | LoginProfile: 146 | Password: GenAIBuilder! 147 | PasswordResetRequired: false 148 | Type: AWS::IAM::User 149 | User02: 150 | Properties: 151 | UserName: User02 152 | Groups: 153 | - !Ref 'IAMGroup' 154 | LoginProfile: 155 | Password: GenAIBuilder! 156 | PasswordResetRequired: false 157 | Type: AWS::IAM::User 158 | User03: 159 | Properties: 160 | UserName: User03 161 | Groups: 162 | - !Ref 'IAMGroup' 163 | LoginProfile: 164 | Password: GenAIBuilder! 165 | PasswordResetRequired: false 166 | Type: AWS::IAM::User 167 | User04: 168 | Properties: 169 | UserName: User04 170 | Groups: 171 | - !Ref 'IAMGroup' 172 | LoginProfile: 173 | Password: GenAIBuilder! 174 | PasswordResetRequired: false 175 | Type: AWS::IAM::User 176 | User05: 177 | Properties: 178 | UserName: User05 179 | Groups: 180 | - !Ref 'IAMGroup' 181 | LoginProfile: 182 | Password: GenAIBuilder! 183 | PasswordResetRequired: false 184 | Type: AWS::IAM::User 185 | User06: 186 | Properties: 187 | UserName: User06 188 | Groups: 189 | - !Ref 'IAMGroup' 190 | LoginProfile: 191 | Password: GenAIBuilder! 192 | PasswordResetRequired: false 193 | Type: AWS::IAM::User 194 | User07: 195 | Properties: 196 | UserName: User07 197 | Groups: 198 | - !Ref 'IAMGroup' 199 | LoginProfile: 200 | Password: GenAIBuilder! 201 | PasswordResetRequired: false 202 | Type: AWS::IAM::User 203 | User08: 204 | Properties: 205 | UserName: User08 206 | Groups: 207 | - !Ref 'IAMGroup' 208 | LoginProfile: 209 | Password: GenAIBuilder! 210 | PasswordResetRequired: false 211 | Type: AWS::IAM::User 212 | User09: 213 | Properties: 214 | UserName: User09 215 | Groups: 216 | - !Ref 'IAMGroup' 217 | LoginProfile: 218 | Password: GenAIBuilder! 219 | PasswordResetRequired: false 220 | Type: AWS::IAM::User 221 | User10: 222 | Properties: 223 | UserName: User10 224 | Groups: 225 | - !Ref 'IAMGroup' 226 | LoginProfile: 227 | Password: GenAIBuilder! 228 | PasswordResetRequired: false 229 | Type: AWS::IAM::User 230 | -------------------------------------------------------------------------------- /tools/gen-ai-builder-init-v1.2.yaml: -------------------------------------------------------------------------------- 1 | Metadata: 2 | AWS::CloudFormation::Interface: 3 | ParameterGroups: [] 4 | ParameterLabels: {} 5 | Comments: Prepares Account User Access for GenAI Builder Event 6 | CreatedBy: Carter Meyers (AWS) 7 | Description: GenAI Builder Event Prep - v1.2 8 | LastUpdated: October 21, 2023 9 | Version: v1.0 10 | Outputs: 11 | User01: 12 | Value: !Join 13 | - '' 14 | - - 'Sign-in URL: https://' 15 | - !Ref 'AWS::AccountId' 16 | - '.signin.aws.amazon.com/console | Username: ' 17 | - !Ref 'User01' 18 | - ' | Password: ' 19 | - GenAIBuilder! 20 | User02: 21 | Value: !Join 22 | - '' 23 | - - 'Sign-in URL: https://' 24 | - !Ref 'AWS::AccountId' 25 | - '.signin.aws.amazon.com/console | Username: ' 26 | - !Ref 'User02' 27 | - ' | Password: ' 28 | - GenAIBuilder! 29 | User03: 30 | Value: !Join 31 | - '' 32 | - - 'Sign-in URL: https://' 33 | - !Ref 'AWS::AccountId' 34 | - '.signin.aws.amazon.com/console | Username: ' 35 | - !Ref 'User03' 36 | - ' | Password: ' 37 | - GenAIBuilder! 38 | User04: 39 | Value: !Join 40 | - '' 41 | - - 'Sign-in URL: https://' 42 | - !Ref 'AWS::AccountId' 43 | - '.signin.aws.amazon.com/console | Username: ' 44 | - !Ref 'User04' 45 | - ' | Password: ' 46 | - GenAIBuilder! 47 | User05: 48 | Value: !Join 49 | - '' 50 | - - 'Sign-in URL: https://' 51 | - !Ref 'AWS::AccountId' 52 | - '.signin.aws.amazon.com/console | Username: ' 53 | - !Ref 'User05' 54 | - ' | Password: ' 55 | - GenAIBuilder! 56 | Resources: 57 | IAMGroup: 58 | Properties: 59 | ManagedPolicyArns: [] 60 | Policies: 61 | - PolicyDocument: 62 | Statement: 63 | - Action: 64 | - sns:* 65 | - cognito-identity:* 66 | - s3:* 67 | - apigateway:* 68 | - cloudformation:* 69 | - cloud9:* 70 | - amplify:* 71 | - iam:* 72 | - kendra:* 73 | - serverlessrepo:* 74 | - lambda:* 75 | - logs:* 76 | - cloudwatch:* 77 | - ec2:* 78 | - cognito-idp:* 79 | - secretsmanager:* 80 | - aws-marketplace:* 81 | - bedrock:* 82 | Effect: Allow 83 | Resource: 84 | - '*' 85 | - Action: 86 | - bedrock:UpdateProvisionedModelThroughput 87 | - bedrock:DeleteProvisionedModelThroughput 88 | - bedrock:CreateProvisionedModelThroughput 89 | Effect: Deny 90 | Resource: 91 | - '*' 92 | PolicyName: gen-ai-builder-access 93 | Type: AWS::IAM::Group 94 | 95 | User01: 96 | Properties: 97 | UserName: User01 98 | Groups: 99 | - !Ref 'IAMGroup' 100 | LoginProfile: 101 | Password: GenAIBuilder! 102 | PasswordResetRequired: false 103 | Type: AWS::IAM::User 104 | User02: 105 | Properties: 106 | UserName: User02 107 | Groups: 108 | - !Ref 'IAMGroup' 109 | LoginProfile: 110 | Password: GenAIBuilder! 111 | PasswordResetRequired: false 112 | Type: AWS::IAM::User 113 | User03: 114 | Properties: 115 | UserName: User03 116 | Groups: 117 | - !Ref 'IAMGroup' 118 | LoginProfile: 119 | Password: GenAIBuilder! 120 | PasswordResetRequired: false 121 | Type: AWS::IAM::User 122 | User04: 123 | Properties: 124 | UserName: User04 125 | Groups: 126 | - !Ref 'IAMGroup' 127 | LoginProfile: 128 | Password: GenAIBuilder! 129 | PasswordResetRequired: false 130 | Type: AWS::IAM::User 131 | User05: 132 | Properties: 133 | UserName: User05 134 | Groups: 135 | - !Ref 'IAMGroup' 136 | LoginProfile: 137 | Password: GenAIBuilder! 138 | PasswordResetRequired: false 139 | Type: AWS::IAM::User 140 | -------------------------------------------------------------------------------- /tools/resize.sh: -------------------------------------------------------------------------------- 1 | 2 | #!/bin/bash 3 | 4 | # Specify the desired volume size in GiB as a command line argument. If not specified, default to 20 GiB. 5 | SIZE=${1:-40} 6 | 7 | # Get the ID of the environment host Amazon EC2 instance. 8 | TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 60") 9 | INSTANCEID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/instance-id 2> /dev/null) 10 | REGION=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/placement/region 2> /dev/null) 11 | 12 | # Get the ID of the Amazon EBS volume associated with the instance. 13 | VOLUMEID=$(aws ec2 describe-instances \ 14 | --instance-id $INSTANCEID \ 15 | --query "Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.VolumeId" \ 16 | --output text \ 17 | --region $REGION) 18 | 19 | # Resize the EBS volume. 20 | aws ec2 modify-volume --volume-id $VOLUMEID --size $SIZE 21 | 22 | # Wait for the resize to finish. 23 | while [ \ 24 | "$(aws ec2 describe-volumes-modifications \ 25 | --volume-id $VOLUMEID \ 26 | --filters Name=modification-state,Values="optimizing","completed" \ 27 | --query "length(VolumesModifications)"\ 28 | --output text)" != "1" ]; do 29 | sleep 1 30 | done 31 | 32 | # Check if we're on an NVMe filesystem 33 | if [[ -e "/dev/xvda" && $(readlink -f /dev/xvda) = "/dev/xvda" ]] 34 | then 35 | # Rewrite the partition table so that the partition takes up all the space that it can. 36 | sudo growpart /dev/xvda 1 37 | # Expand the size of the file system. 38 | # Check if we're on AL2 39 | STR=$(cat /etc/os-release) 40 | SUB="VERSION_ID=\"2\"" 41 | if [[ "$STR" == *"$SUB"* ]] 42 | then 43 | sudo xfs_growfs -d / 44 | else 45 | sudo resize2fs /dev/xvda1 46 | fi 47 | 48 | else 49 | # Rewrite the partition table so that the partition takes up all the space that it can. 50 | sudo growpart /dev/nvme0n1 1 51 | 52 | # Expand the size of the file system. 53 | # Check if we're on AL2 54 | STR=$(cat /etc/os-release) 55 | SUB="VERSION_ID=\"2\"" 56 | if [[ "$STR" == *"$SUB"* ]] 57 | then 58 | sudo xfs_growfs -d / 59 | else 60 | sudo resize2fs /dev/nvme0n1p1 61 | fi 62 | fi 63 | -------------------------------------------------------------------------------- /tools/samconf.toml: -------------------------------------------------------------------------------- 1 | version = 0.1 2 | [default] 3 | [default.deploy] 4 | [default.deploy.parameters] 5 | region = "" 6 | stack_name = "" 7 | resolve_s3 = true 8 | s3_prefix = "" 9 | capabilities = "CAPABILITY_IAM" 10 | parameter_overrides = "StackName=\"\" KendraIndexId=\"\" BedrockWSS3Bucket=\"\" ApiGatewayStageName=\"prod\"" 11 | -------------------------------------------------------------------------------- /tools/solutions/kbfunction.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import boto3 4 | 5 | import traceback 6 | 7 | 8 | region = boto3.session.Session().region_name 9 | KB_ID = os.environ["KB_ID"] 10 | 11 | 12 | def lambda_handler(event, context): 13 | boto3_version = boto3.__version__ 14 | print(f"Boto3 version: {boto3_version}") 15 | 16 | print(f"Event is: {event}") 17 | event_body = json.loads(event["body"]) 18 | prompt = event_body["query"] 19 | model_id = event_body["model_id"] 20 | 21 | response = '' 22 | status_code = 200 23 | 24 | try: 25 | model_arn = 'arn:aws:bedrock:'+region+'::foundation-model/'+model_id 26 | print(f"Model arn: {model_arn}") 27 | 28 | response = retrieveAndGenerate(prompt, model_arn)["output"]["text"] 29 | return { 30 | 'statusCode': status_code, 31 | 'headers': { 32 | 'Access-Control-Allow-Origin': '*', 33 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 34 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 35 | }, 36 | 'body': json.dumps({'answer': response}) 37 | } 38 | 39 | except Exception as e: 40 | print(f"An unexpected error occurred: {str(e)}") 41 | stack_trace = traceback.format_exc() 42 | print(stack_trace) 43 | return { 44 | 'statusCode': status_code, 45 | 'headers': { 46 | 'Access-Control-Allow-Origin': '*', 47 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 48 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 49 | }, 50 | 'body': json.dumps({'error': str(e)}) 51 | } 52 | 53 | def retrieveAndGenerate(prompt, model_arn): 54 | bedrock_agent_runtime = boto3.client( 55 | service_name = "bedrock-agent-runtime") 56 | return bedrock_agent_runtime.retrieve_and_generate( 57 | input={ 58 | 'text': prompt 59 | }, 60 | retrieveAndGenerateConfiguration={ 61 | 'type': 'KNOWLEDGE_BASE', 62 | 'knowledgeBaseConfiguration': { 63 | 'knowledgeBaseId': KB_ID, 64 | 'modelArn': model_arn 65 | } 66 | } 67 | ) 68 | -------------------------------------------------------------------------------- /tools/solutions/llmfunction.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import json 3 | import traceback 4 | 5 | 6 | region = boto3.session.Session().region_name 7 | 8 | def lambda_handler(event, context): 9 | boto3_version = boto3.__version__ 10 | print(f"Boto3 version: {boto3_version}") 11 | 12 | print(f"Event is: {event}") 13 | event_body = json.loads(event["body"]) 14 | prompt = event_body["query"] 15 | temperature = event_body["temperature"] 16 | max_tokens = event_body["max_tokens"] 17 | model_id = event_body["model_id"] 18 | 19 | response = '' 20 | status_code = 200 21 | 22 | try: 23 | if model_id == 'mistral.mistral-7b-instruct-v0:2': 24 | response = invoke_mistral_7b(model_id, prompt, temperature, max_tokens) 25 | elif model_id == 'meta.llama3-1-8b-instruct-v1:0': 26 | response = invoke_llama(model_id, prompt, temperature, max_tokens) 27 | else: 28 | response = invoke_claude(model_id, prompt, temperature, max_tokens) 29 | 30 | return { 31 | 'statusCode': status_code, 32 | 'headers': { 33 | 'Access-Control-Allow-Origin': '*', 34 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 35 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 36 | }, 37 | 'body': json.dumps({'answer': response}) 38 | } 39 | 40 | except Exception as e: 41 | print(f"An unexpected error occurred: {str(e)}") 42 | stack_trace = traceback.format_exc() 43 | print(stack_trace) 44 | return { 45 | 'statusCode': status_code, 46 | 'headers': { 47 | 'Access-Control-Allow-Origin': '*', 48 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 49 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 50 | }, 51 | 'body': json.dumps({'error': str(e)}) 52 | } 53 | 54 | 55 | def invoke_claude(model_id, prompt, temperature, max_tokens): 56 | try: 57 | 58 | instruction = f"Human: {prompt} nAssistant:" 59 | bedrock_runtime_client = boto3.client(service_name="bedrock-runtime", region_name=region) 60 | body= { 61 | "anthropic_version": "bedrock-2023-05-31", 62 | "max_tokens": max_tokens, 63 | "messages": [ 64 | { 65 | "role": "user", 66 | "content": [{"type": "text", "text": prompt}], 67 | } 68 | ], 69 | } 70 | 71 | response = bedrock_runtime_client.invoke_model( 72 | modelId=model_id, body=json.dumps(body) 73 | ) 74 | 75 | response_body = json.loads(response["body"].read()) 76 | outputs = response_body.get("content") 77 | completions = [output["text"] for output in outputs] 78 | print(f"completions: {completions[0]}") 79 | 80 | return completions[0] 81 | 82 | except Exception as e: 83 | raise 84 | 85 | def invoke_mistral_7b(model_id, prompt, temperature, max_tokens): 86 | try: 87 | instruction = f"[INST] {prompt} [/INST]" 88 | bedrock_runtime_client = boto3.client(service_name="bedrock-runtime", region_name=region) 89 | 90 | body = { 91 | "prompt": instruction, 92 | "max_tokens": max_tokens, 93 | "temperature": temperature, 94 | } 95 | 96 | response = bedrock_runtime_client.invoke_model( 97 | modelId=model_id, body=json.dumps(body) 98 | ) 99 | response_body = json.loads(response["body"].read()) 100 | outputs = response_body.get("outputs") 101 | print(f"response: {outputs}") 102 | 103 | completions = [output["text"] for output in outputs] 104 | return completions[0] 105 | except Exception as e: 106 | raise 107 | 108 | def invoke_llama(model_id, prompt, temperature, max_tokens): 109 | print(f"Invoking llam model {model_id}" ) 110 | print(f"max_tokens {max_tokens}" ) 111 | try: 112 | instruction = f"[INST]You are a very intelligent bot with exceptional critical thinking, help me answering below question.[/INST]" 113 | total_prompt = f"{instruction}\n{prompt}" 114 | 115 | print(f"Prompt template {total_prompt}" ) 116 | 117 | bedrock_runtime_client = boto3.client(service_name="bedrock-runtime", region_name=region) 118 | 119 | body = { 120 | "prompt": total_prompt, 121 | "max_gen_len": max_tokens, 122 | "temperature": temperature, 123 | "top_p": 0.9 124 | } 125 | 126 | response = bedrock_runtime_client.invoke_model( 127 | modelId=model_id, body=json.dumps(body) 128 | ) 129 | response_body = json.loads(response["body"].read()) 130 | print(f"response: {response_body}") 131 | return response_body ['generation'] 132 | except Exception as e: 133 | raise 134 | -------------------------------------------------------------------------------- /tools/solutions/ragfunction.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | import boto3 4 | from langchain_community.retrievers import AmazonKendraRetriever 5 | from langchain_aws import ChatBedrock 6 | from langchain.chains import ConversationalRetrievalChain 7 | from langchain.memory import ConversationBufferMemory 8 | from langchain.prompts import PromptTemplate 9 | from langchain.chains import RetrievalQA 10 | 11 | import traceback 12 | 13 | kendra = boto3.client('kendra') 14 | chain_type = 'stuff' 15 | 16 | KENDRA_INDEX_ID = os.getenv('KENDRA_INDEX_ID') 17 | S3_BUCKET_NAME = os.environ["S3_BUCKET_NAME"] 18 | 19 | 20 | refine_prompt_template = ( 21 | "Below is an instruction that describes a task. " 22 | "Write a response that appropriately completes the request.\n\n" 23 | "### Instruction:\n" 24 | "This is the original question: {question}\n" 25 | "The existing answer: {existing_answer}\n" 26 | "Now there are some additional texts, (if needed) you can use them to improve your existing answer." 27 | "\n\n" 28 | "{context_str}\n" 29 | "\\nn" 30 | "Please use the new passage to further improve your answer.\n\n" 31 | "### Response: " 32 | ) 33 | 34 | initial_qa_template = ( 35 | "Below is an instruction that describes a task. " 36 | "Write a response that appropriately completes the request.\n\n" 37 | "### Instruction:\n" 38 | "The following is background knowledge：\n" 39 | "{context_str}" 40 | "\n" 41 | "Please answer this question based on the background knowledge provided above：{question}。\n\n" 42 | "### Response: " 43 | ) 44 | 45 | def lambda_handler(event, context): 46 | print(f"Event is: {event}") 47 | 48 | event_body = json.loads(event["body"]) 49 | question = event_body["query"] 50 | print(f"Query is: {question}") 51 | 52 | model_id = event_body["model_id"] 53 | temperature = event_body["temperature"] 54 | max_tokens = event_body["max_tokens"] 55 | 56 | response = '' 57 | status_code = 200 58 | 59 | PROMPT_TEMPLATE = 'prompt-engineering/claude-prompt-template.txt' 60 | 61 | try: 62 | if model_id == 'mistral.mistral-7b-instruct-v0:2': 63 | llm = get_mistral_llm(model_id,temperature,max_tokens) 64 | PROMPT_TEMPLATE = 'prompt-engineering/mistral-prompt-template.txt' 65 | elif model_id == 'meta.llama3-1-8b-instruct-v1:0': 66 | llm = get_llama_llm(model_id,temperature,max_tokens) 67 | PROMPT_TEMPLATE = 'prompt-engineering/llama-prompt-template.txt' 68 | else: 69 | llm = get_claude_llm(model_id,temperature,max_tokens) 70 | PROMPT_TEMPLATE = 'prompt-engineering/claude-prompt-template.txt' 71 | 72 | # Read the prompt template from S3 bucket 73 | s3 = boto3.resource('s3') 74 | obj = s3.Object(S3_BUCKET_NAME, PROMPT_TEMPLATE) 75 | prompt_template = obj.get()['Body'].read().decode('utf-8') 76 | print(f"prompt template: {prompt_template}") 77 | 78 | retriever = AmazonKendraRetriever(kendra_client=kendra,index_id=KENDRA_INDEX_ID) 79 | 80 | 81 | if chain_type == "stuff": 82 | PROMPT = PromptTemplate( 83 | template=prompt_template, input_variables=["context", "question"] 84 | ) 85 | chain_type_kwargs = {"prompt": PROMPT} 86 | qa = RetrievalQA.from_chain_type( 87 | llm=llm, 88 | chain_type="stuff", 89 | retriever=retriever, 90 | return_source_documents=True, 91 | chain_type_kwargs=chain_type_kwargs) 92 | response = qa(question, return_only_outputs=False) 93 | elif chain_type == "refine": 94 | refine_prompt = PromptTemplate( 95 | input_variables=["question", "existing_answer", "context_str"], 96 | template=refine_prompt_template, 97 | ) 98 | initial_qa_prompt = PromptTemplate( 99 | input_variables=["context_str", "question"], 100 | template=prompt_template, 101 | ) 102 | chain_type_kwargs = {"question_prompt": initial_qa_prompt, "refine_prompt": refine_prompt} 103 | qa = RetrievalQA.from_chain_type( 104 | llm=llm, 105 | chain_type="refine", 106 | retriever=retriever, 107 | return_source_documents=True, 108 | chain_type_kwargs=chain_type_kwargs) 109 | response = qa(question, return_only_outputs=False) 110 | 111 | print('Response', response) 112 | source_documents = response.get('source_documents') 113 | source_docs = [] 114 | previous_source = None 115 | previous_score = None 116 | response_data = [] 117 | 118 | #if chain_type == "stuff": 119 | for source_doc in source_documents: 120 | source = source_doc.metadata['source'] 121 | score = source_doc.metadata["score"] 122 | if source != previous_source or score != previous_score: 123 | source_data = { 124 | "source": source, 125 | "score": score 126 | } 127 | response_data.append(source_data) 128 | previous_source = source 129 | previous_score = score 130 | 131 | response_with_metadata = { 132 | "answer": response.get('result'), 133 | "source_documents": response_data 134 | } 135 | 136 | return { 137 | 'statusCode': status_code, 138 | 'headers': { 139 | 'Access-Control-Allow-Origin': '*', 140 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 141 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 142 | }, 143 | 'body': json.dumps(response_with_metadata) 144 | } 145 | 146 | 147 | except Exception as e: 148 | print(f"An unexpected error occurred: {str(e)}") 149 | stack_trace = traceback.format_exc() 150 | print(f"stack trace: {stack_trace}") 151 | print(f"error: {str(e)}") 152 | 153 | response = str(e) 154 | return { 155 | 'statusCode': status_code, 156 | 'headers': { 157 | 'Access-Control-Allow-Origin': '*', 158 | 'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token', 159 | 'Access-Control-Allow-Methods': 'OPTIONS,POST' 160 | }, 161 | 'body': json.dumps({'error': response}) 162 | } 163 | 164 | def get_claude_llm(model_id, temperature, max_tokens): 165 | model_kwargs = { 166 | "max_tokens": max_tokens, 167 | "temperature": temperature, 168 | "top_k": 50, 169 | "top_p": 0.95 170 | } 171 | llm = ChatBedrock(model_id=model_id, model_kwargs=model_kwargs) 172 | return llm 173 | 174 | def get_llama_llm(model_id, temperature, max_tokens): 175 | model_kwargs = { 176 | "max_gen_len": max_tokens, 177 | "temperature": temperature, 178 | "top_p": 0.9 179 | } 180 | llm = ChatBedrock(model_id=model_id, model_kwargs=model_kwargs) 181 | return llm 182 | 183 | def get_mistral_llm(model_id, temperature, max_tokens): 184 | model_kwargs = { 185 | "max_tokens": max_tokens, 186 | "temperature": temperature, 187 | "top_k": 50, 188 | "top_p": 0.9 189 | } 190 | llm = ChatBedrock(model_id=model_id, model_kwargs=model_kwargs) 191 | return llm 192 | 193 | def get_memory(): 194 | 195 | memory = ConversationBufferMemory( 196 | memory_key="chat_history", 197 | input_key="question", 198 | output_key="answer", 199 | return_messages=True 200 | ) 201 | return memory 202 | -------------------------------------------------------------------------------- /tools/troubleshoot.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | #### FIRT RUN ALL THESE ### 4 | export CFNStackName=bedrock-ws954 5 | export S3BucketName=$(aws cloudformation describe-stacks --stack-name ${CFNStackName} --query "Stacks[0].Outputs[?OutputKey=='S3BucketName'].OutputValue" --output text) 6 | export AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r '.region') 7 | export KendraIndexID=$(aws cloudformation describe-stacks --stack-name ${CFNStackName} --query "Stacks[0].Outputs[?OutputKey=='KendraIndexID'].OutputValue" --output text) 8 | export SAMStackName="sam-$CFNStackName" 9 | export BedrockApiUrl=$(aws cloudformation describe-stacks --stack-name ${SAMStackName} --query "Stacks[0].Outputs[?OutputKey=='BedrockApiUrl'].OutputValue" --output text) 10 | export SecretName=$(aws cloudformation describe-stacks --stack-name ${SAMStackName} --query "Stacks[0].Outputs[?OutputKey=='SecretsName'].OutputValue" --output text) 11 | 12 | 13 | #Amplify and sam builds 14 | #nvm use 16 15 | #node --version 16 | 17 | 18 | export PATH=~/.npm-global/bin:$PATH 19 | 20 | #update Ampliyfy and build frontend 21 | cd ~/environment/bedrock-serverless-workshop/frontend 22 | npm run build 23 | 24 | rm -r build 25 | mv dist build 26 | amplify publish --yes 27 | 28 | ### SAM deployments 29 | cd ~/environment/bedrock-serverless-workshop 30 | sam build && sam deploy 31 | 32 | aws secretsmanager get-secret-value --secret-id $SecretName | jq -r .SecretString 33 | 34 | export KB_ID=$(aws bedrock-agent list-knowledge-bases | jq -r '.knowledgeBaseSummaries[0].knowledgeBaseId') 35 | echo "Knowledge Base ID: $KB_ID" 36 | sed -Ei "s|copy_kb_id|${KB_ID}|g" ./template.yaml 37 | 38 | --------------------------------------------------------------------------------