├── pipeline ├── modules │ ├── repo │ │ ├── data.tf │ │ ├── variables.tf │ │ ├── main.tf │ │ ├── outputs.tf │ │ └── README.md │ ├── kms │ │ ├── data.tf │ │ ├── outputs.tf │ │ ├── variables.tf │ │ ├── README.md │ │ └── main.tf │ ├── s3 │ │ ├── provider.tf │ │ ├── variables.tf │ │ ├── outputs.tf │ │ ├── README.md │ │ └── main.tf │ ├── iam-role │ │ ├── data.tf │ │ ├── outputs.tf │ │ ├── variables.tf │ │ ├── README.md │ │ └── main.tf │ ├── codepipeline │ │ ├── outputs.tf │ │ ├── variables.tf │ │ ├── README.md │ │ └── main.tf │ └── codebuild │ │ ├── outputs.tf │ │ ├── main.tf │ │ ├── variables.tf │ │ └── README.md ├── providers.tf ├── data.tf ├── locals.tf ├── examples │ └── terraform.tfvars ├── LICENSE ├── templates │ ├── buildspec_destroy.yml │ ├── buildspec_plan.yml │ ├── buildspec_apply.yml │ ├── buildspec_validate.yml │ └── scripts │ │ └── tf_ssp_validation.sh ├── outputs.tf ├── variables.tf ├── main.tf └── README.md ├── modules ├── guardrails │ ├── variables.tf │ ├── outputs.tf │ └── main.tf ├── s3 │ ├── variables.tf │ ├── data.tf │ ├── outputs.tf │ └── main.tf ├── bedrock-agent │ ├── instruction.txt │ ├── outputs.tf │ ├── data.tf │ ├── locals.tf │ ├── variables.tf │ └── main.tf ├── lambda │ ├── outputs.tf │ ├── data.tf │ ├── locals.tf │ ├── variables.tf │ ├── files │ │ └── index.py │ └── main.tf ├── custom-model │ ├── outputs.tf │ ├── data.tf │ ├── variables.tf │ ├── locals.tf │ ├── bank_qa.py │ └── main.tf ├── opensearch │ ├── data.tf │ ├── outputs.tf │ ├── providers.tf │ ├── variables.tf │ └── main.tf ├── knowledge-base │ ├── outputs.tf │ ├── variables.tf │ └── main.tf └── iam │ ├── locals.tf │ ├── data.tf │ ├── outputs.tf │ ├── variables.tf │ └── main.tf ├── instruction.txt ├── images ├── Custom-Model-Deployment.png └── ML-Model-Deployment-Page-2.drawio.png ├── CODE_OF_CONDUCT.md ├── .gitignore ├── .terraform-docs.yaml ├── data.tf ├── locals.tf ├── providers.tf ├── .pre-commit-config.yaml ├── LICENSE ├── templates ├── buildspec_destroy.yml ├── buildspec_plan.yml ├── buildspec_apply.yml ├── buildspec_validate.yml └── scripts │ └── tf_ssp_validation.sh ├── variables.tf ├── .header.md ├── CONTRIBUTING.md ├── main.tf └── README.md /pipeline/modules/repo/data.tf: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /modules/guardrails/variables.tf: -------------------------------------------------------------------------------- 1 | variable "name" { 2 | default = "guardrail_name" 3 | type = string 4 | } -------------------------------------------------------------------------------- /instruction.txt: -------------------------------------------------------------------------------- 1 | You are an assisant who can provide detailed information about Banking FAQ to a current or prospective customer. -------------------------------------------------------------------------------- /modules/s3/variables.tf: -------------------------------------------------------------------------------- 1 | variable "bucket_name" { 2 | type = string 3 | description = "Name of the S3 bucket to be created" 4 | } -------------------------------------------------------------------------------- /modules/bedrock-agent/instruction.txt: -------------------------------------------------------------------------------- 1 | You are an assistant that provides the data about events. A user may ask you the information about events. -------------------------------------------------------------------------------- /images/Custom-Model-Deployment.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/bedrock-custom-model-knowledge-base-terraform/HEAD/images/Custom-Model-Deployment.png -------------------------------------------------------------------------------- /images/ML-Model-Deployment-Page-2.drawio.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/bedrock-custom-model-knowledge-base-terraform/HEAD/images/ML-Model-Deployment-Page-2.drawio.png -------------------------------------------------------------------------------- /modules/bedrock-agent/outputs.tf: -------------------------------------------------------------------------------- 1 | output "agent_id" { 2 | description = "The ID of the AWS CloudControl Agent resource." 3 | value = awscc_bedrock_agent.sample_asst.agent_id 4 | } -------------------------------------------------------------------------------- /modules/lambda/outputs.tf: -------------------------------------------------------------------------------- 1 | output "lambda_arn" { 2 | description = "The Amazon Resource Name (ARN) of the Lambda function." 3 | value = aws_lambda_function.sample_api.arn 4 | } -------------------------------------------------------------------------------- /modules/custom-model/outputs.tf: -------------------------------------------------------------------------------- 1 | output "custom_model_arn" { 2 | description = "The ARN of the custom model" 3 | value = aws_bedrock_custom_model.cm_cohere_v1.custom_model_arn 4 | } -------------------------------------------------------------------------------- /modules/s3/data.tf: -------------------------------------------------------------------------------- 1 | # Use data sources to get common information about the environment 2 | data "aws_caller_identity" "this" {} 3 | data "aws_partition" "this" {} 4 | data "aws_region" "this" {} -------------------------------------------------------------------------------- /modules/opensearch/data.tf: -------------------------------------------------------------------------------- 1 | # Use data sources to get common information about the environment 2 | data "aws_caller_identity" "this" {} 3 | data "aws_partition" "this" {} 4 | data "aws_region" "this" {} -------------------------------------------------------------------------------- /pipeline/providers.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_version = "~> 1.0" 3 | required_providers { 4 | aws = { 5 | source = "hashicorp/aws" 6 | version = "~> 5.48" 7 | } 8 | } 9 | 10 | } 11 | -------------------------------------------------------------------------------- /modules/s3/outputs.tf: -------------------------------------------------------------------------------- 1 | output "arn" { 2 | description = "arn of the S3 bucket created" 3 | value = aws_s3_bucket.this.arn 4 | } 5 | 6 | output "bucket_name" { 7 | description = "Name of the S3 bucket created" 8 | value = aws_s3_bucket.this.id 9 | } -------------------------------------------------------------------------------- /modules/custom-model/data.tf: -------------------------------------------------------------------------------- 1 | data "aws_caller_identity" "this" {} 2 | data "aws_partition" "this" {} 3 | data "aws_region" "this" {} 4 | 5 | data "aws_bedrock_foundation_model" "cohere_command_light_text_v14" { 6 | model_id = "cohere.command-light-text-v14:7:4k" 7 | } -------------------------------------------------------------------------------- /modules/custom-model/variables.tf: -------------------------------------------------------------------------------- 1 | variable "custom_model_bucket" { 2 | type = string 3 | description = "Name of the bucket used" 4 | } 5 | 6 | variable "bedrock_custom_role_arn" { 7 | type = string 8 | description = "ARN of the custom role used with Bedrock" 9 | } -------------------------------------------------------------------------------- /modules/bedrock-agent/data.tf: -------------------------------------------------------------------------------- 1 | # Use data sources to get common information about the environment 2 | data "aws_caller_identity" "this" {} 3 | data "aws_partition" "this" {} 4 | data "aws_region" "this" {} 5 | 6 | data "aws_bedrock_foundation_model" "agent" { 7 | model_id = var.agent_model_id 8 | } -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /modules/guardrails/outputs.tf: -------------------------------------------------------------------------------- 1 | output "guardrail_identifier" { 2 | description = "Identifier of the Bedrock Guardrail" 3 | value = awscc_bedrock_guardrail.sample.id 4 | } 5 | 6 | output "guardrail_version" { 7 | description = "Version of the Bedrock Guardrail" 8 | value = awscc_bedrock_guardrail_version.sample.version 9 | } -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .idea 3 | .venv 4 | 5 | # Local .terraform directories 6 | **/.terraform/* 7 | **/.terraform 8 | 9 | # .tfstate files 10 | *.tfstate 11 | *.tfstate.* 12 | 13 | # Crash log files 14 | crash.log 15 | crash.*.log 16 | 17 | # Ignore CLI configuration files 18 | .terraformrc 19 | terraform.rc 20 | .terraform.lock.hcl -------------------------------------------------------------------------------- /modules/knowledge-base/outputs.tf: -------------------------------------------------------------------------------- 1 | output "kb_id" { 2 | description = "The ID of the Bedrock Agent Knowledge Base" 3 | value = aws_bedrockagent_knowledge_base.sample_kb.id 4 | } 5 | 6 | output "kb_arn" { 7 | description = "The ARN of the Bedrock Agent Knowledge Base" 8 | value = aws_bedrockagent_knowledge_base.sample_kb.arn 9 | } -------------------------------------------------------------------------------- /.terraform-docs.yaml: -------------------------------------------------------------------------------- 1 | formatter: markdown 2 | header-from: .header.md 3 | settings: 4 | anchor: true 5 | color: true 6 | default: true 7 | escape: true 8 | html: true 9 | indent: 2 10 | required: true 11 | sensitive: true 12 | type: true 13 | 14 | sort: 15 | enabled: true 16 | by: required 17 | 18 | output: 19 | file: README.md 20 | mode: replace -------------------------------------------------------------------------------- /data.tf: -------------------------------------------------------------------------------- 1 | # Use data sources to get common information about the environment 2 | data "aws_caller_identity" "this" {} 3 | data "aws_partition" "this" {} 4 | data "aws_region" "this" {} 5 | 6 | data "aws_bedrock_foundation_model" "kb" { 7 | model_id = var.kb_model_id 8 | } 9 | 10 | data "aws_iam_policy" "lambda_basic_execution" { 11 | name = "AWSLambdaVPCAccessExecutionRole" 12 | } -------------------------------------------------------------------------------- /modules/opensearch/outputs.tf: -------------------------------------------------------------------------------- 1 | output "opensearch_collection_arn" { 2 | description = "The name of the collection for the Knowledge Base Open Source Software (OSS) content" 3 | value = aws_opensearchserverless_collection.sample_kb.arn 4 | } 5 | 6 | 7 | output "opensearch_index_name" { 8 | description = "The name of the OpenSearch index" 9 | value = opensearch_index.sample_kb.name 10 | } -------------------------------------------------------------------------------- /locals.tf: -------------------------------------------------------------------------------- 1 | locals { 2 | account_id = data.aws_caller_identity.this.account_id 3 | partition = data.aws_partition.this.partition 4 | region = data.aws_region.this.name 5 | region_name_tokenized = split("-", local.region) 6 | region_short = "${substr(local.region_name_tokenized[0], 0, 2)}${substr(local.region_name_tokenized[1], 0, 1)}${local.region_name_tokenized[2]}" 7 | } -------------------------------------------------------------------------------- /modules/opensearch/providers.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_providers { 3 | aws = { 4 | source = "hashicorp/aws" 5 | version = "~> 5.48" 6 | } 7 | opensearch = { 8 | source = "opensearch-project/opensearch" 9 | version = "= 2.2.0" 10 | } 11 | awscc = { 12 | source = "hashicorp/awscc" 13 | version = "~> 1.0" 14 | } 15 | } 16 | required_version = "~> 1.0" 17 | } -------------------------------------------------------------------------------- /modules/iam/locals.tf: -------------------------------------------------------------------------------- 1 | locals { 2 | account_id = data.aws_caller_identity.this.account_id 3 | partition = data.aws_partition.this.partition 4 | region = data.aws_region.this.name 5 | region_name_tokenized = split("-", local.region) 6 | region_short = "${substr(local.region_name_tokenized[0], 0, 2)}${substr(local.region_name_tokenized[1], 0, 1)}${local.region_name_tokenized[2]}" 7 | } -------------------------------------------------------------------------------- /modules/lambda/data.tf: -------------------------------------------------------------------------------- 1 | # Use data sources to get common information about the environment 2 | data "aws_caller_identity" "this" {} 3 | data "aws_partition" "this" {} 4 | data "aws_region" "this" {} 5 | 6 | data "archive_file" "sample_api_zip" { 7 | type = "zip" 8 | source_file = "${path.module}/files/index.py" 9 | output_path = "${path.module}/files/sample_api.zip" 10 | output_file_mode = "0666" 11 | } -------------------------------------------------------------------------------- /modules/lambda/locals.tf: -------------------------------------------------------------------------------- 1 | locals { 2 | account_id = data.aws_caller_identity.this.account_id 3 | partition = data.aws_partition.this.partition 4 | region = data.aws_region.this.name 5 | region_name_tokenized = split("-", local.region) 6 | region_short = "${substr(local.region_name_tokenized[0], 0, 2)}${substr(local.region_name_tokenized[1], 0, 1)}${local.region_name_tokenized[2]}" 7 | } -------------------------------------------------------------------------------- /modules/bedrock-agent/locals.tf: -------------------------------------------------------------------------------- 1 | locals { 2 | account_id = data.aws_caller_identity.this.account_id 3 | partition = data.aws_partition.this.partition 4 | region = data.aws_region.this.name 5 | region_name_tokenized = split("-", local.region) 6 | region_short = "${substr(local.region_name_tokenized[0], 0, 2)}${substr(local.region_name_tokenized[1], 0, 1)}${local.region_name_tokenized[2]}" 7 | } -------------------------------------------------------------------------------- /modules/custom-model/locals.tf: -------------------------------------------------------------------------------- 1 | locals { 2 | account_id = data.aws_caller_identity.this.account_id 3 | partition = data.aws_partition.this.partition 4 | region = data.aws_region.this.name 5 | region_name_tokenized = split("-", local.region) 6 | region_short = "${substr(local.region_name_tokenized[0], 0, 2)}${substr(local.region_name_tokenized[1], 0, 1)}${local.region_name_tokenized[2]}" 7 | } -------------------------------------------------------------------------------- /modules/iam/data.tf: -------------------------------------------------------------------------------- 1 | # Use data sources to get common information about the environment 2 | data "aws_caller_identity" "this" {} 3 | data "aws_partition" "this" {} 4 | data "aws_region" "this" {} 5 | 6 | data "aws_iam_policy_document" "bedrock_custom_policy" { 7 | statement { 8 | sid = "AllowS3Access" 9 | actions = ["s3:GetObject", "s3:PutObject", "s3:ListBucket"] 10 | resources = [var.custom_model_s3_arn, "${var.custom_model_s3_arn}/*"] 11 | } 12 | } 13 | 14 | -------------------------------------------------------------------------------- /modules/opensearch/variables.tf: -------------------------------------------------------------------------------- 1 | variable "kb_oss_collection_name" { 2 | type = string 3 | description = "The name of the collection for the Knowledge Base Open Source Software (OSS) content." 4 | } 5 | 6 | variable "bedrock_role_arn" { 7 | type = string 8 | description = "The Amazon Resource Name (ARN) of the IAM role that grants permissions to the Bedrock resources." 9 | } 10 | 11 | variable "index_name" { 12 | type = string 13 | description = "The name of the OpenSearch index" 14 | } -------------------------------------------------------------------------------- /pipeline/data.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | data "aws_caller_identity" "current" {} 8 | data "aws_region" "current" {} -------------------------------------------------------------------------------- /pipeline/modules/kms/data.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | data "aws_caller_identity" "current" {} 8 | data "aws_region" "current" {} -------------------------------------------------------------------------------- /pipeline/modules/s3/provider.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | provider "aws" { 8 | alias = "replication" 9 | region = "us-east-2" 10 | } -------------------------------------------------------------------------------- /pipeline/locals.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | locals { 8 | account_id = data.aws_caller_identity.current.account_id 9 | region = data.aws_region.current.name 10 | } -------------------------------------------------------------------------------- /pipeline/modules/kms/outputs.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | output "arn" { 8 | value = aws_kms_key.encryption_key.arn 9 | description = "The ARN of the KMS key" 10 | } -------------------------------------------------------------------------------- /modules/iam/outputs.tf: -------------------------------------------------------------------------------- 1 | output "bedrock_role_arn" { 2 | description = "ARN of the Bedrock IAM role" 3 | value = aws_iam_role.bedrock_kb_sample_kb.arn 4 | } 5 | 6 | output "bedrock_role_name" { 7 | description = "Name of the Bedrock IAM role" 8 | value = aws_iam_role.bedrock_kb_sample_kb.name 9 | } 10 | 11 | output "lambda_role_arn" { 12 | description = "ARN of the Lambda IAM role" 13 | value = aws_iam_role.lambda_sample_api.arn 14 | } 15 | 16 | output "bedrock_custom_role_arn" { 17 | description = "ARN of the bedrock custom model IAM role" 18 | value = aws_iam_role.bedrock_custom_role.arn 19 | } -------------------------------------------------------------------------------- /pipeline/modules/repo/variables.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | variable "provider_type" { 8 | type = string 9 | description = "Name of the external provider you are connecting to" 10 | } -------------------------------------------------------------------------------- /providers.tf: -------------------------------------------------------------------------------- 1 | terraform { 2 | required_providers { 3 | aws = { 4 | source = "hashicorp/aws" 5 | version = "~> 5.48" 6 | } 7 | opensearch = { 8 | source = "opensearch-project/opensearch" 9 | version = "= 2.2.0" 10 | } 11 | awscc = { 12 | source = "hashicorp/awscc" 13 | version = "~> 1.0" 14 | } 15 | random = { 16 | source = "hashicorp/random" 17 | version = "~>3.0" 18 | } 19 | } 20 | backend "s3" { 21 | bucket = "statetest-s3" 22 | key = "pipeline" 23 | region = "us-east-1" 24 | } 25 | required_version = "~> 1.0" 26 | } 27 | 28 | 29 | -------------------------------------------------------------------------------- /pipeline/modules/repo/main.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | resource "aws_codestarconnections_connection" "bedrock-cn" { 8 | name = "bedrock-test-github-connection" 9 | provider_type = var.provider_type 10 | } -------------------------------------------------------------------------------- /modules/lambda/variables.tf: -------------------------------------------------------------------------------- 1 | variable "action_group_name" { 2 | type = string 3 | description = "The name of the action group for monitoring alerts." 4 | } 5 | 6 | variable "lambda_role_arn" { 7 | type = string 8 | description = "The Amazon Resource Name (ARN) of the IAM role for the Lambda function." 9 | } 10 | 11 | variable "subnet_ids" { 12 | type = list(string) 13 | description = "The list of subnet IDs where the Lambda function will be deployed." 14 | } 15 | 16 | variable "security_group_ids" { 17 | type = list(string) 18 | description = "The list of security group IDs to be associated with the Lambda function." 19 | } -------------------------------------------------------------------------------- /pipeline/modules/repo/outputs.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | output "connection_arn" { 8 | value = aws_codestarconnections_connection.bedrock-cn.arn 9 | description = "List containing the arn of the Repository" 10 | } 11 | -------------------------------------------------------------------------------- /modules/custom-model/bank_qa.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | from datasets import load_dataset 4 | 5 | # Load the dataset 6 | dataset = load_dataset("SohamNale/Banking_Dataset_for_LLM_Finetuning", split="train") 7 | 8 | with open("banking_qa.jsonl", "w", encoding="utf-8") as jsonl_file: 9 | total_rows = len(dataset) 10 | 11 | # Iterate through the dataset 12 | for index, row in enumerate(dataset, start=1): 13 | qa_pair = { 14 | "prompt": row["question"], 15 | "completion": row["answer"] 16 | } 17 | json.dump(qa_pair, jsonl_file, ensure_ascii=False) 18 | if index < total_rows: 19 | jsonl_file.write("\n") 20 | -------------------------------------------------------------------------------- /modules/iam/variables.tf: -------------------------------------------------------------------------------- 1 | variable "kb_name" { 2 | description = "The knowledge base name." 3 | type = string 4 | default = "sample" 5 | } 6 | 7 | variable "agent_name" { 8 | description = "The agent name." 9 | type = string 10 | default = "sampleAssistant" 11 | } 12 | 13 | variable "action_group_name" { 14 | description = "The action group name." 15 | type = string 16 | default = "sampleAPI" 17 | } 18 | 19 | variable "lambda_iam_policy" { 20 | type = string 21 | } 22 | 23 | variable "s3_arn" { 24 | type = string 25 | } 26 | 27 | variable "kb_model_arn" { 28 | type = string 29 | } 30 | 31 | variable "custom_model_s3_arn" { 32 | type = string 33 | } -------------------------------------------------------------------------------- /pipeline/modules/kms/variables.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | variable "tags" { 8 | description = "Tags to be attached to the KMS Key" 9 | type = map(any) 10 | } 11 | 12 | variable "codepipeline_role_arn" { 13 | description = "ARN of the codepipeline IAM role" 14 | type = string 15 | } -------------------------------------------------------------------------------- /pipeline/modules/iam-role/data.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | data "aws_region" "current" {} 8 | data "aws_caller_identity" "current" {} 9 | data "aws_partition" "current" {} 10 | 11 | # To be used only in case of an Existing Repository 12 | data "aws_iam_role" "existing_codepipeline_role" { 13 | count = var.create_new_role ? 0 : 1 14 | name = var.codepipeline_iam_role_name 15 | } -------------------------------------------------------------------------------- /modules/guardrails/main.tf: -------------------------------------------------------------------------------- 1 | #guardrail 2 | resource "awscc_bedrock_guardrail" "sample" { 3 | name = var.name 4 | blocked_input_messaging = "Blocked input" 5 | blocked_outputs_messaging = "Blocked output" 6 | description = "sample guardrail" 7 | 8 | content_policy_config = { 9 | filters_config = [ 10 | { 11 | input_strength = "MEDIUM" 12 | output_strength = "MEDIUM" 13 | type = "HATE" 14 | }, 15 | { 16 | input_strength = "HIGH" 17 | output_strength = "HIGH" 18 | type = "VIOLENCE" 19 | } 20 | ] 21 | } 22 | } 23 | 24 | resource "awscc_bedrock_guardrail_version" "sample" { 25 | guardrail_identifier = awscc_bedrock_guardrail.sample.guardrail_id 26 | description = "sample guardrail version" 27 | } -------------------------------------------------------------------------------- /modules/knowledge-base/variables.tf: -------------------------------------------------------------------------------- 1 | variable "kb_name" { 2 | type = string 3 | description = "The name of the Knowledge Base" 4 | } 5 | 6 | variable "s3_arn" { 7 | type = string 8 | description = "The ARN of the S3 bucket" 9 | } 10 | 11 | variable "bedrock_role_arn" { 12 | type = string 13 | description = "The ARN of the Bedrock role" 14 | } 15 | 16 | variable "bedrock_role_name" { 17 | type = string 18 | description = "The name of the Bedrock role" 19 | } 20 | 21 | variable "kb_model_arn" { 22 | type = string 23 | description = "The ARN of the Knowledge Base model" 24 | } 25 | 26 | variable "opensearch_arn" { 27 | type = string 28 | description = "The ARN of the OpenSearch domain" 29 | } 30 | 31 | variable "opensearch_index_name" { 32 | type = string 33 | description = "The name of the OpenSearch index" 34 | } -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | fail_fast: false 3 | minimum_pre_commit_version: "2.6.0" 4 | repos: 5 | - repo: https://github.com/terraform-docs/terraform-docs 6 | rev: 212db41760d7fc45d736d5eb94a483d0d2a12049 # frozen: v0.16.0 To update run: `pre-commit autoupdate --freeze` 7 | hooks: 8 | - id: terraform-docs-go 9 | args: 10 | - "--config=.terraform-docs.yaml" 11 | - "--lockfile=false" 12 | - "./" 13 | - repo: https://github.com/antonbabenko/pre-commit-terraform 14 | rev: v1.81.0 15 | hooks: 16 | - id: terraform_fmt 17 | - repo: meta 18 | hooks: 19 | - id: check-useless-excludes 20 | - repo: https://github.com/pre-commit/pre-commit-hooks 21 | rev: v4.4.0 22 | hooks: 23 | - id: trailing-whitespace 24 | - id: check-added-large-files 25 | - id: check-json 26 | - id: check-yaml 27 | - id: detect-private-key -------------------------------------------------------------------------------- /pipeline/modules/iam-role/outputs.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | output "role_arn" { 8 | value = var.create_new_role ? aws_iam_role.codepipeline_role[0].arn : data.aws_iam_role.existing_codepipeline_role[0].arn 9 | description = "The ARN of the IAM Role" 10 | } 11 | 12 | output "role_name" { 13 | value = var.create_new_role ? aws_iam_role.codepipeline_role[0].name : data.aws_iam_role.existing_codepipeline_role[0].name 14 | description = "The ARN of the IAM Role" 15 | } 16 | -------------------------------------------------------------------------------- /pipeline/modules/codepipeline/outputs.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | output "id" { 8 | value = aws_codepipeline.terraform_pipeline.id 9 | description = "The id of the CodePipeline" 10 | } 11 | 12 | output "name" { 13 | value = aws_codepipeline.terraform_pipeline.name 14 | description = "The name of the CodePipeline" 15 | } 16 | 17 | output "arn" { 18 | value = aws_codepipeline.terraform_pipeline.arn 19 | description = "The arn of the CodePipeline" 20 | } 21 | 22 | 23 | -------------------------------------------------------------------------------- /pipeline/examples/terraform.tfvars: -------------------------------------------------------------------------------- 1 | project_name = "bedrock" 2 | environment = "dev" 3 | source_repo_name = "aws-samples/bedrock-custom-model-knowledge-base-terraform" 4 | source_repo_branch = "main" 5 | provider_type = "GitHub" 6 | create_new_role = true 7 | #codepipeline_iam_role_name = - Use this to specify the role name to be used by codepipeline if the create_new_role flag is set to false. 8 | stage_input = [ 9 | { name = "validate", category = "Test", owner = "AWS", provider = "CodeBuild", input_artifacts = "SourceOutput", output_artifacts = "ValidateOutput" }, 10 | { name = "plan", category = "Test", owner = "AWS", provider = "CodeBuild", input_artifacts = "ValidateOutput", output_artifacts = "PlanOutput" }, 11 | { name = "apply", category = "Build", owner = "AWS", provider = "CodeBuild", input_artifacts = "PlanOutput", output_artifacts = "ApplyOutput" } 12 | ] 13 | build_projects = ["validate", "plan", "apply"] 14 | -------------------------------------------------------------------------------- /pipeline/LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 7 | the Software, and to permit persons to whom the Software is furnished to do so. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 10 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 11 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 12 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 13 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 14 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 15 | 16 | -------------------------------------------------------------------------------- /pipeline/modules/codebuild/outputs.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | output "id" { 8 | value = aws_codebuild_project.terraform_codebuild_project[*].id 9 | description = "List of IDs of the CodeBuild projects" 10 | } 11 | 12 | output "name" { 13 | value = aws_codebuild_project.terraform_codebuild_project[*].name 14 | description = "List of Names of the CodeBuild projects" 15 | } 16 | 17 | output "arn" { 18 | value = aws_codebuild_project.terraform_codebuild_project[*].arn 19 | description = "List of ARNs of the CodeBuild projects" 20 | } 21 | 22 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT No Attribution 2 | 3 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 13 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 14 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 15 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 16 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 17 | 18 | -------------------------------------------------------------------------------- /pipeline/modules/s3/variables.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | variable "project_name" { 8 | description = "Name of the project to be prefixed to create the s3 bucket" 9 | type = string 10 | } 11 | variable "tags" { 12 | description = "Tags to be associated with the S3 bucket" 13 | type = map(any) 14 | } 15 | 16 | variable "kms_key_arn" { 17 | description = "ARN of KMS key for encryption" 18 | type = string 19 | } 20 | 21 | variable "codepipeline_role_arn" { 22 | description = "ARN of the codepipeline IAM role" 23 | type = string 24 | } -------------------------------------------------------------------------------- /pipeline/modules/s3/outputs.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | output "arn" { 8 | value = aws_s3_bucket.codepipeline_bucket.arn 9 | description = "The ARN of the S3 Bucket" 10 | } 11 | 12 | output "bucket" { 13 | value = aws_s3_bucket.codepipeline_bucket.bucket 14 | description = "The Name of the S3 Bucket" 15 | } 16 | 17 | output "bucket_url" { 18 | value = "https://s3.console.aws.amazon.com/s3/buckets/${aws_s3_bucket.codepipeline_bucket.bucket}?region=${aws_s3_bucket.codepipeline_bucket.region}&tab=objects" 19 | description = "The URL of the S3 Bucket" 20 | } 21 | -------------------------------------------------------------------------------- /templates/buildspec_destroy.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | TF_VERSION: "1.10.0" 13 | 14 | phases: 15 | install: 16 | runtime-versions: 17 | python: 3.9 18 | commands: 19 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 20 | - "unzip -o terraform.zip" 21 | - "mv terraform /bin" 22 | - "rm terraform.zip" 23 | 24 | build: 25 | commands: 26 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 27 | - "terraform destroy --auto-approve" 28 | 29 | artifacts: 30 | files: 31 | - '**/*' 32 | -------------------------------------------------------------------------------- /pipeline/templates/buildspec_destroy.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | TF_VERSION: "1.10.0" 13 | 14 | phases: 15 | install: 16 | runtime-versions: 17 | python: 3.9 18 | commands: 19 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 20 | - "unzip -o terraform.zip" 21 | - "mv terraform /bin" 22 | - "rm terraform.zip" 23 | 24 | build: 25 | commands: 26 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 27 | - "terraform destroy --auto-approve" 28 | 29 | artifacts: 30 | files: 31 | - '**/*' 32 | -------------------------------------------------------------------------------- /templates/buildspec_plan.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | TF_VERSION: "1.10.0" 13 | 14 | phases: 15 | install: 16 | runtime-versions: 17 | python: 3.9 18 | commands: 19 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 20 | - "unzip -o terraform.zip" 21 | - "mv terraform /bin" 22 | - "rm terraform.zip" 23 | build: 24 | commands: 25 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 26 | - "echo ## TERRAFORM PLAN : Generate the Terraform Plan" 27 | - "terraform plan -out tfapply" 28 | artifacts: 29 | files: 30 | - '**/*' 31 | -------------------------------------------------------------------------------- /pipeline/templates/buildspec_plan.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | TF_VERSION: "1.10.0" 13 | 14 | phases: 15 | install: 16 | runtime-versions: 17 | python: 3.9 18 | commands: 19 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 20 | - "unzip -o terraform.zip" 21 | - "mv terraform /bin" 22 | - "rm terraform.zip" 23 | build: 24 | commands: 25 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 26 | - "echo ## TERRAFORM PLAN : Generate the Terraform Plan" 27 | - "terraform plan -out tfapply" 28 | artifacts: 29 | files: 30 | - '**/*' 31 | -------------------------------------------------------------------------------- /templates/buildspec_apply.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | TF_VERSION: "1.10.0" 13 | 14 | phases: 15 | install: 16 | runtime-versions: 17 | python: 3.9 18 | commands: 19 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 20 | - "unzip -o terraform.zip" 21 | - "mv terraform /bin" 22 | - "rm terraform.zip" 23 | build: 24 | commands: 25 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 26 | - "echo ## TERRAFORM APPLY : Starting with the Terraform Apply" 27 | - "terraform apply -input=false tfapply" 28 | 29 | artifacts: 30 | files: 31 | - '**/*' -------------------------------------------------------------------------------- /pipeline/templates/buildspec_apply.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | TF_VERSION: "1.10.0" 13 | 14 | phases: 15 | install: 16 | runtime-versions: 17 | python: 3.9 18 | commands: 19 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 20 | - "unzip -o terraform.zip" 21 | - "mv terraform /bin" 22 | - "rm terraform.zip" 23 | build: 24 | commands: 25 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 26 | - "echo ## TERRAFORM APPLY : Starting with the Terraform Apply" 27 | - "terraform apply -input=false tfapply" 28 | 29 | artifacts: 30 | files: 31 | - '**/*' -------------------------------------------------------------------------------- /modules/lambda/files/index.py: -------------------------------------------------------------------------------- 1 | import json 2 | import logging 3 | import os 4 | 5 | logger = logging.getLogger() 6 | log_level = os.environ.get("log_level", logging.INFO) 7 | 8 | logger.setLevel(log_level) 9 | logger.info("Log level set to %s" % logger.getEffectiveLevel()) 10 | 11 | def lambda_handler(event, _): 12 | logger.debug(json.dumps(event)) 13 | # agent = event['agent'] 14 | actionGroup = event['actionGroup'] 15 | function = event['function'] 16 | # parameters = event.get('parameters', []) 17 | 18 | # Execute your business logic here. For more information, refer to: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-lambda.html 19 | responseBody = { 20 | "TEXT": { 21 | "body": "The function {} was called successfully!".format(function) 22 | } 23 | } 24 | 25 | action_response = { 26 | 'actionGroup': actionGroup, 27 | 'function': function, 28 | 'functionResponse': { 29 | 'responseBody': responseBody 30 | } 31 | 32 | } 33 | 34 | dummy_function_response = {'response': action_response, 'messageVersion': event['messageVersion']} 35 | logger.info("Response: {}".format(dummy_function_response)) 36 | 37 | return dummy_function_response 38 | -------------------------------------------------------------------------------- /pipeline/modules/iam-role/variables.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | variable "source_repository_name" { 8 | type = string 9 | description = "Name of the Source CodeCommit repository" 10 | } 11 | 12 | variable "project_name" { 13 | description = "Unique name for this project" 14 | type = string 15 | } 16 | 17 | variable "codepipeline_iam_role_name" { 18 | description = "Name of the IAM role to be used by the project" 19 | type = string 20 | } 21 | 22 | variable "tags" { 23 | description = "Tags to be attached to the IAM Role" 24 | type = map(any) 25 | } 26 | 27 | variable "kms_key_arn" { 28 | description = "ARN of KMS key for encryption" 29 | type = string 30 | } 31 | 32 | variable "s3_bucket_arn" { 33 | description = "The ARN of the S3 Bucket" 34 | type = string 35 | } 36 | 37 | variable "create_new_role" { 38 | type = bool 39 | description = "Flag for deciding if a new role needs to be created" 40 | default = true 41 | } -------------------------------------------------------------------------------- /pipeline/modules/codebuild/main.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | resource "aws_codebuild_project" "terraform_codebuild_project" { 8 | 9 | count = length(var.build_projects) 10 | 11 | name = "${var.project_name}-${var.build_projects[count.index]}" 12 | service_role = var.role_arn 13 | encryption_key = var.kms_key_arn 14 | tags = var.tags 15 | artifacts { 16 | type = var.build_project_source 17 | } 18 | environment { 19 | compute_type = var.builder_compute_type 20 | image = var.builder_image 21 | type = var.builder_type 22 | privileged_mode = true 23 | image_pull_credentials_type = var.builder_image_pull_credentials_type 24 | } 25 | logs_config { 26 | cloudwatch_logs { 27 | status = "ENABLED" 28 | } 29 | } 30 | source { 31 | type = var.build_project_source 32 | buildspec = "./templates/buildspec_${var.build_projects[count.index]}.yml" 33 | } 34 | } -------------------------------------------------------------------------------- /modules/lambda/main.tf: -------------------------------------------------------------------------------- 1 | # Action group Lambda function 2 | resource "aws_lambda_function" "sample_api" { 3 | #checkov:skip=CKV_AWS_116: "Ensure that AWS Lambda function is configured for a Dead Letter Queue(DLQ)" 4 | #checkov:skip=CKV_AWS_115: "Ensure that AWS Lambda function is configured for function-level concurrent execution limit" 5 | #checkov:skip=CKV_AWS_272: "Ensure AWS Lambda function is configured to validate code-signing" 6 | function_name = var.action_group_name 7 | role = var.lambda_role_arn 8 | description = "A Lambda function for the action group ${var.action_group_name}" 9 | filename = data.archive_file.sample_api_zip.output_path 10 | handler = "index.lambda_handler" 11 | runtime = "python3.12" 12 | # source_code_hash is required to detect changes to Lambda code/zip 13 | source_code_hash = data.archive_file.sample_api_zip.output_base64sha256 14 | tracing_config { 15 | mode = "Active" 16 | } 17 | vpc_config { 18 | subnet_ids = var.subnet_ids 19 | security_group_ids = var.security_group_ids 20 | } 21 | } 22 | 23 | resource "aws_lambda_permission" "sample_api" { 24 | action = "lambda:invokeFunction" 25 | function_name = aws_lambda_function.sample_api.function_name 26 | principal = "bedrock.amazonaws.com" 27 | source_account = local.account_id 28 | source_arn = "arn:${local.partition}:bedrock:${local.region}:${local.account_id}:agent/*" 29 | } -------------------------------------------------------------------------------- /modules/bedrock-agent/variables.tf: -------------------------------------------------------------------------------- 1 | variable "agent_model_id" { 2 | description = "The ID of the foundational model used by the agent." 3 | type = string 4 | default = "anthropic.claude-3-haiku-20240307-v1:0" 5 | } 6 | 7 | variable "agent_name" { 8 | description = "The name of the agent." 9 | type = string 10 | } 11 | 12 | variable "kb_id" { 13 | description = "The ID of the knowledge base associated with the agent." 14 | type = string 15 | } 16 | 17 | variable "guardrail_identifier" { 18 | description = "The identifier of the guardrail used by the agent." 19 | type = string 20 | } 21 | 22 | variable "guardrail_version" { 23 | description = "The version of the guardrail used by the agent." 24 | type = string 25 | } 26 | 27 | variable "action_group_lambda_arn" { 28 | description = "The Amazon Resource Name (ARN) of the Lambda function associated with the action group." 29 | type = string 30 | } 31 | 32 | variable "action_group_name" { 33 | description = "The name of the action group." 34 | type = string 35 | default = "sampleAPI" 36 | } 37 | 38 | variable "action_group_desc" { 39 | description = "The description of the action group." 40 | type = string 41 | default = "An assistant that provides sample information." 42 | } 43 | 44 | variable "kb_arn" { 45 | description = "The Amazon Resource Name (ARN) of the knowledge base associated with the agent." 46 | type = string 47 | } -------------------------------------------------------------------------------- /pipeline/modules/kms/README.md: -------------------------------------------------------------------------------- 1 | 2 | ## Requirements 3 | 4 | No requirements. 5 | 6 | ## Providers 7 | 8 | | Name | Version | 9 | |------|---------| 10 | | [aws](#provider\_aws) | n/a | 11 | 12 | ## Modules 13 | 14 | No modules. 15 | 16 | ## Resources 17 | 18 | | Name | Type | 19 | |------|------| 20 | | [aws_kms_key.encryption_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource | 21 | | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | 22 | | [aws_iam_policy_document.kms_key_policy_doc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | 23 | | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | 24 | 25 | ## Inputs 26 | 27 | | Name | Description | Type | Default | Required | 28 | |------|-------------|------|---------|:--------:| 29 | | [codepipeline\_role\_arn](#input\_codepipeline\_role\_arn) | ARN of the codepipeline IAM role | `string` | n/a | yes | 30 | | [tags](#input\_tags) | Tags to be attached to the KMS Key | `map(any)` | n/a | yes | 31 | 32 | ## Outputs 33 | 34 | | Name | Description | 35 | |------|-------------| 36 | | [arn](#output\_arn) | The ARN of the KMS key | 37 | -------------------------------------------------------------------------------- /modules/s3/main.tf: -------------------------------------------------------------------------------- 1 | # S3 bucket for the knowledge base 2 | #tfsec:ignore:aws-s3-enable-bucket-logging 3 | resource "aws_s3_bucket" "this" { 4 | bucket = var.bucket_name 5 | force_destroy = true 6 | } 7 | 8 | resource "aws_s3_bucket_public_access_block" "this" { 9 | bucket = aws_s3_bucket.this.id 10 | block_public_acls = true 11 | ignore_public_acls = true 12 | block_public_policy = true 13 | restrict_public_buckets = true 14 | } 15 | 16 | resource "aws_kms_key" "this" { 17 | description = "KMS key to encrypt S3 bucket" 18 | enable_key_rotation = true 19 | deletion_window_in_days = 7 20 | policy = jsonencode({ 21 | Version = "2012-10-17" 22 | Id = "key-default-1" 23 | Statement = [ 24 | { 25 | Sid = "Enable IAM User Permissions" 26 | Effect = "Allow" 27 | Principal = { 28 | AWS = "arn:${data.aws_partition.this.partition}:iam::${data.aws_caller_identity.this.account_id}:root" 29 | }, 30 | Action = "kms:*" 31 | Resource = "*" 32 | } 33 | ] 34 | }) 35 | } 36 | 37 | resource "aws_s3_bucket_server_side_encryption_configuration" "this" { 38 | bucket = aws_s3_bucket.this.id 39 | rule { 40 | apply_server_side_encryption_by_default { 41 | kms_master_key_id = aws_kms_key.this.arn 42 | sse_algorithm = "aws:kms" 43 | } 44 | } 45 | } 46 | 47 | resource "aws_s3_bucket_versioning" "this" { 48 | bucket = aws_s3_bucket.this.id 49 | versioning_configuration { 50 | status = "Enabled" 51 | } 52 | depends_on = [aws_s3_bucket_server_side_encryption_configuration.this] 53 | } -------------------------------------------------------------------------------- /pipeline/outputs.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | output "codebuild_name" { 8 | value = module.codebuild_terraform.name 9 | description = "The Name of the Codebuild Project" 10 | } 11 | 12 | output "codebuild_arn" { 13 | value = module.codebuild_terraform.arn 14 | description = "The ARN of the Codebuild Project" 15 | } 16 | 17 | output "codepipeline_name" { 18 | value = module.codepipeline_terraform.name 19 | description = "The Name of the CodePipeline" 20 | } 21 | 22 | output "codepipeline_arn" { 23 | value = module.codepipeline_terraform.arn 24 | description = "The ARN of the CodePipeline" 25 | } 26 | 27 | output "iam_arn" { 28 | value = module.codepipeline_iam_role.role_arn 29 | description = "The ARN of the IAM Role used by the CodePipeline" 30 | } 31 | 32 | output "kms_arn" { 33 | value = module.codepipeline_kms.arn 34 | description = "The ARN of the KMS key used in the codepipeline" 35 | } 36 | 37 | output "s3_arn" { 38 | value = module.s3_artifacts_bucket.arn 39 | description = "The ARN of the S3 Bucket" 40 | } 41 | 42 | output "s3_bucket_name" { 43 | value = module.s3_artifacts_bucket.bucket 44 | description = "The Name of the S3 Bucket" 45 | } 46 | -------------------------------------------------------------------------------- /pipeline/modules/codepipeline/variables.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | variable "project_name" { 8 | description = "Unique name for this project" 9 | type = string 10 | } 11 | 12 | variable "source_repo_name" { 13 | description = "Source repo name of the CodeCommit repository" 14 | type = string 15 | } 16 | 17 | variable "source_repo_branch" { 18 | description = "Default branch in the Source repo for which CodePipeline needs to be configured" 19 | type = string 20 | } 21 | 22 | variable "s3_bucket_name" { 23 | description = "S3 bucket name to be used for storing the artifacts" 24 | type = string 25 | } 26 | 27 | variable "codepipeline_role_arn" { 28 | description = "ARN of the codepipeline IAM role" 29 | type = string 30 | } 31 | 32 | variable "kms_key_arn" { 33 | description = "ARN of KMS key for encryption" 34 | type = string 35 | } 36 | 37 | variable "tags" { 38 | description = "Tags to be attached to the CodePipeline" 39 | type = map(any) 40 | } 41 | 42 | variable "stages" { 43 | description = "List of Map containing information about the stages of the CodePipeline" 44 | type = list(map(any)) 45 | } 46 | 47 | variable "connection_arn" { 48 | description = "Connection ARN" 49 | type = string 50 | } -------------------------------------------------------------------------------- /modules/custom-model/main.tf: -------------------------------------------------------------------------------- 1 | resource "aws_s3_object" "v1_training_fine_tune" { 2 | bucket = var.custom_model_bucket 3 | key = "training_data_v1/banking_qa.jsonl" 4 | source = "banking_qa.jsonl" 5 | } 6 | 7 | # resource "terraform_data" "training_data_fine_tune_v1" { 8 | # input = "banking_qa.jsonl" 9 | 10 | # provisioner "local-exec" { 11 | # command = "pip3 install datasets && python ${path.module}/bank_qa.py" 12 | # } 13 | # } 14 | 15 | resource "time_sleep" "iam_consistency_delay" { 16 | create_duration = "120s" 17 | depends_on = [aws_s3_object.v1_training_fine_tune] 18 | } 19 | 20 | resource "aws_bedrock_custom_model" "cm_cohere_v1" { 21 | custom_model_name = "cm_cohere_v001-${local.region_short}-${local.account_id}" 22 | job_name = "cm.command-light-text-v14.v001--${local.region_short}-${local.account_id}-${random_integer.suffix.result}" 23 | base_model_identifier = data.aws_bedrock_foundation_model.cohere_command_light_text_v14.model_arn 24 | role_arn = var.bedrock_custom_role_arn 25 | customization_type = "FINE_TUNING" 26 | 27 | hyperparameters = { 28 | "epochCount" = "1" 29 | "batchSize" = "8" 30 | "learningRate" = "0.00001" 31 | "earlyStoppingPatience" = "6" 32 | "earlyStoppingThreshold" = "0.01" 33 | "evalPercentage" = "20.0" 34 | } 35 | 36 | output_data_config { 37 | s3_uri = "s3://${var.custom_model_bucket}/output_data_v1/" 38 | } 39 | 40 | training_data_config { 41 | s3_uri = "s3://${var.custom_model_bucket}/training_data_v1/banking_qa.jsonl" 42 | } 43 | depends_on = [aws_s3_object.v1_training_fine_tune] 44 | } 45 | 46 | resource "random_integer" "suffix" { 47 | min = 23 48 | max = 50000 49 | 50 | } 51 | -------------------------------------------------------------------------------- /modules/knowledge-base/main.tf: -------------------------------------------------------------------------------- 1 | resource "aws_iam_role_policy" "bedrock_kb_sample_kb_model" { 2 | name = "AmazonBedrockOSSPolicyForKnowledgeBase_${var.kb_name}" 3 | role = var.bedrock_role_name 4 | policy = jsonencode({ 5 | Version = "2012-10-17" 6 | Statement = [ 7 | { 8 | Action = ["aoss:*"] 9 | Effect = "Allow" 10 | Resource = [var.opensearch_arn] 11 | } 12 | ] 13 | }) 14 | } 15 | 16 | resource "time_sleep" "iam_consistency_delay" { 17 | create_duration = "120s" 18 | depends_on = [aws_iam_role_policy.bedrock_kb_sample_kb_model] 19 | } 20 | 21 | 22 | resource "aws_bedrockagent_knowledge_base" "sample_kb" { 23 | name = var.kb_name 24 | role_arn = var.bedrock_role_arn 25 | knowledge_base_configuration { 26 | vector_knowledge_base_configuration { 27 | embedding_model_arn = var.kb_model_arn 28 | } 29 | type = "VECTOR" 30 | } 31 | storage_configuration { 32 | type = "OPENSEARCH_SERVERLESS" 33 | opensearch_serverless_configuration { 34 | collection_arn = var.opensearch_arn 35 | vector_index_name = var.opensearch_index_name 36 | field_mapping { 37 | vector_field = "bedrock-knowledge-base-default-vector" 38 | text_field = "AMAZON_BEDROCK_TEXT_CHUNK" 39 | metadata_field = "AMAZON_BEDROCK_METADATA" 40 | } 41 | } 42 | } 43 | depends_on = [time_sleep.iam_consistency_delay, aws_iam_role_policy.bedrock_kb_sample_kb_model] 44 | } 45 | 46 | resource "aws_bedrockagent_data_source" "sample_kb" { 47 | knowledge_base_id = aws_bedrockagent_knowledge_base.sample_kb.id 48 | name = "${var.kb_name}DataSource" 49 | data_source_configuration { 50 | type = "S3" 51 | s3_configuration { 52 | bucket_arn = var.s3_arn 53 | } 54 | } 55 | } -------------------------------------------------------------------------------- /variables.tf: -------------------------------------------------------------------------------- 1 | variable "kb_s3_bucket_name_prefix" { 2 | description = "The name prefix of the S3 bucket for the data source of the knowledge base." 3 | type = string 4 | default = "sample-kb" 5 | } 6 | 7 | variable "kb_oss_collection_name" { 8 | description = "The name of the OpenSearch Service (OSS) collection for the knowledge base." 9 | type = string 10 | default = "bedrock-sample-kb" 11 | } 12 | 13 | variable "kb_model_id" { 14 | description = "The ID of the foundational model used by the knowledge base." 15 | type = string 16 | default = "amazon.titan-embed-text-v1" 17 | } 18 | 19 | variable "kb_name" { 20 | description = "The name of the knowledge base." 21 | type = string 22 | default = "sample" 23 | } 24 | 25 | variable "agent_model_id" { 26 | description = "The ID of the foundational model used by the agent." 27 | type = string 28 | default = "anthropic.claude-3-haiku-20240307-v1:0" 29 | } 30 | 31 | variable "agent_name" { 32 | description = "The name of the agent." 33 | type = string 34 | default = "sampleAssistant" 35 | } 36 | 37 | variable "agent_desc" { 38 | description = "The description of the agent." 39 | type = string 40 | default = "An assistant that provides sample information." 41 | } 42 | 43 | variable "action_group_name" { 44 | description = "The name of the action group." 45 | type = string 46 | default = "sampleAPI" 47 | } 48 | 49 | variable "action_group_desc" { 50 | description = "The description of the action group." 51 | type = string 52 | default = "An action group that provides sample information." 53 | } 54 | 55 | variable "subnet_ids" { 56 | type = list(string) 57 | description = "The list of subnet IDs where the Lambda function will be deployed." 58 | default = ["subnet-123"] 59 | } 60 | 61 | variable "security_group_ids" { 62 | type = list(string) 63 | description = "The list of security group IDs to be associated with the Lambda function." 64 | default = ["sg-123"] 65 | } -------------------------------------------------------------------------------- /pipeline/modules/codebuild/variables.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | variable "project_name" { 8 | description = "Unique name for this project" 9 | type = string 10 | } 11 | 12 | variable "role_arn" { 13 | description = "Codepipeline IAM role arn. " 14 | type = string 15 | default = "" 16 | } 17 | 18 | variable "s3_bucket_name" { 19 | description = "Name of the S3 bucket used to store the deployment artifacts" 20 | type = string 21 | } 22 | 23 | variable "tags" { 24 | description = "Tags to be applied to the codebuild project" 25 | type = map(any) 26 | } 27 | 28 | variable "build_projects" { 29 | description = "List of Names of the CodeBuild projects to be created" 30 | type = list(string) 31 | } 32 | 33 | variable "builder_compute_type" { 34 | description = "Information about the compute resources the build project will use" 35 | type = string 36 | } 37 | 38 | variable "builder_image" { 39 | description = "Docker image to use for the build project" 40 | type = string 41 | } 42 | 43 | variable "builder_type" { 44 | description = "Type of build environment to use for related builds" 45 | type = string 46 | } 47 | 48 | variable "builder_image_pull_credentials_type" { 49 | description = "Type of credentials AWS CodeBuild uses to pull images in your build." 50 | type = string 51 | } 52 | 53 | variable "build_project_source" { 54 | description = "Information about the build output artifact location" 55 | type = string 56 | } 57 | 58 | variable "kms_key_arn" { 59 | description = "ARN of KMS key for encryption" 60 | type = string 61 | } -------------------------------------------------------------------------------- /pipeline/modules/codepipeline/README.md: -------------------------------------------------------------------------------- 1 | 2 | ## Requirements 3 | 4 | No requirements. 5 | 6 | ## Providers 7 | 8 | | Name | Version | 9 | |------|---------| 10 | | [aws](#provider\_aws) | n/a | 11 | 12 | ## Modules 13 | 14 | No modules. 15 | 16 | ## Resources 17 | 18 | | Name | Type | 19 | |------|------| 20 | | [aws_codepipeline.terraform_pipeline](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codepipeline) | resource | 21 | 22 | ## Inputs 23 | 24 | | Name | Description | Type | Default | Required | 25 | |------|-------------|------|---------|:--------:| 26 | | [codepipeline\_role\_arn](#input\_codepipeline\_role\_arn) | ARN of the codepipeline IAM role | `string` | n/a | yes | 27 | | [kms\_key\_arn](#input\_kms\_key\_arn) | ARN of KMS key for encryption | `string` | n/a | yes | 28 | | [project\_name](#input\_project\_name) | Unique name for this project | `string` | n/a | yes | 29 | | [s3\_bucket\_name](#input\_s3\_bucket\_name) | S3 bucket name to be used for storing the artifacts | `string` | n/a | yes | 30 | | [source\_repo\_branch](#input\_source\_repo\_branch) | Default branch in the Source repo for which CodePipeline needs to be configured | `string` | n/a | yes | 31 | | [source\_repo\_name](#input\_source\_repo\_name) | Source repo name of the CodeCommit repository | `string` | n/a | yes | 32 | | [stages](#input\_stages) | List of Map containing information about the stages of the CodePipeline | `list(map(any))` | n/a | yes | 33 | | [tags](#input\_tags) | Tags to be attached to the CodePipeline | `map(any)` | n/a | yes | 34 | 35 | ## Outputs 36 | 37 | | Name | Description | 38 | |------|-------------| 39 | | [arn](#output\_arn) | The arn of the CodePipeline | 40 | | [id](#output\_id) | The id of the CodePipeline | 41 | | [name](#output\_name) | The name of the CodePipeline | 42 | -------------------------------------------------------------------------------- /pipeline/modules/codepipeline/main.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | resource "aws_codepipeline" "terraform_pipeline" { 8 | 9 | name = "${var.project_name}-pipeline" 10 | role_arn = var.codepipeline_role_arn 11 | tags = var.tags 12 | 13 | artifact_store { 14 | location = var.s3_bucket_name 15 | type = "S3" 16 | encryption_key { 17 | id = var.kms_key_arn 18 | type = "KMS" 19 | } 20 | } 21 | 22 | stage { 23 | name = "Source" 24 | 25 | action { 26 | name = "Source" 27 | category = "Source" 28 | owner = "AWS" 29 | provider = "CodeStarSourceConnection" 30 | version = "1" 31 | output_artifacts = ["SourceOutput"] 32 | 33 | configuration = { 34 | ConnectionArn = var.connection_arn 35 | FullRepositoryId = var.source_repo_name 36 | BranchName = var.source_repo_branch # Replace with your desired branch 37 | } 38 | } 39 | } 40 | 41 | dynamic "stage" { 42 | for_each = var.stages 43 | 44 | content { 45 | name = "Stage-${stage.value["name"]}" 46 | action { 47 | category = stage.value["category"] 48 | name = "Action-${stage.value["name"]}" 49 | owner = stage.value["owner"] 50 | provider = stage.value["provider"] 51 | input_artifacts = [stage.value["input_artifacts"]] 52 | output_artifacts = [stage.value["output_artifacts"]] 53 | version = "1" 54 | run_order = index(var.stages, stage.value) + 2 55 | 56 | configuration = { 57 | ProjectName = stage.value["provider"] == "CodeBuild" ? "${var.project_name}-${stage.value["name"]}" : null 58 | } 59 | } 60 | } 61 | } 62 | 63 | } -------------------------------------------------------------------------------- /.header.md: -------------------------------------------------------------------------------- 1 | # Custom Bedrock model deployment 2 | 3 | ![](./images/ML-Model-Deployment-Page-2.drawio.png) 4 | 5 | This repository contains sample code demonstrating how to use Terraform to provision a custom model using Bedrock with a dataset which answers some common banking questions for an existing or prospective customer. 6 | 7 | The root Terraform module configuration provisions the below components into a subnet of your choice with the necessary security group attached. 8 | 9 | * Amazon Bedrock 10 | * Bedrock Custom model training job and a custom model on successful run of the training job. 11 | * Bedrock Guardrail 12 | * Bedrock Knowledge Base 13 | * Bedrock Agents 14 | * Amazon Opensearch Collection and Index 15 | * Amazon S3 16 | * AWS IAM 17 | 18 | > [!IMPORTANT] 19 | > A destroy operation on the stack doesn't delete the custom model training job, but rather stops it. Keep in mind that that the name of the job has to be unique on an account. We are using the random provider to generate a custom name for the new deployments of the stack. 20 | 21 | 22 | The solution uses the dataset `SohamNale/Banking_Dataset_for_LLM_Finetuning` and generates training data in the form of a jsonl file for the format below. 23 | 24 | ``` 25 | { 26 | "prompt": "What is a bank account?", 27 | "completion": "A bank account is a deposit account held at a financial institution that allows a customer to store money and perform financial transactions." 28 | } 29 | 30 | ``` 31 | 32 | The training dataset is uploaded to an S3 bucket created as part of the stack and referenced as as the training data for the custome model training job. The job ,when finished fine tuning the foundation model ,creates a custom model which can be used using a provisioned throughput with a no commitment or term based commitment based on your choice. 33 | 34 | ## Prerequisites 35 | 36 | You need to have an AWS account and an AWS Identity and Access Management (IAM) role and user with permissions to create and manage the necessary resources and components for this application. Before proceeding, if you have not previously done so, you must request access to the following Amazon Bedrock models: 37 | 38 | * Amazon: amazon.titan-embed-text-v1 39 | * Anthropic: anthropic.claude-3-haiku-20240307-v1:0 40 | -------------------------------------------------------------------------------- /templates/buildspec_validate.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | SCRIPT_DIR: "./templates/scripts" 13 | TF_VERSION: "1.10.0" 14 | ENVIRONMENT: "dev" 15 | SKIPVALIDATIONFAILURE: "Y" 16 | ENABLE_TFVALIDATE: "Y" 17 | ENABLE_TFFORMAT: "Y" 18 | ENABLE_TFCHECKOV: "Y" 19 | ENABLE_TFSEC: "Y" 20 | TFSEC_VERSION: "v1.28.1" 21 | 22 | phases: 23 | install: 24 | runtime-versions: 25 | python: 3.9 26 | golang: 1.14 27 | commands: 28 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 29 | - "unzip -o terraform.zip" 30 | - "mv terraform /bin" 31 | - "rm terraform.zip" 32 | - "yum update -y" 33 | - "yum install -y curl jq awscli" 34 | - "pip3 install checkov" 35 | - "yum install -y golang" 36 | - "wget -q -O tfsec https://github.com/aquasecurity/tfsec/releases/download/${TFSEC_VERSION}/tfsec-linux-amd64" 37 | - "chmod +x ./tfsec" 38 | - "mv ./tfsec /usr/local/bin/tfsec" 39 | build: 40 | commands: 41 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 42 | - "echo ## TERRAFORM INIT : Initialize the Terraform Workspace" 43 | - "terraform init" 44 | # - "echo ## VALIDATION : Starting ..." 45 | # - "mkdir -p ${CODEBUILD_SRC_DIR}/reports" 46 | # - "/bin/bash ${CODEBUILD_SRC_DIR}/${SCRIPT_DIR}/tf_ssp_validation.sh ${SKIPVALIDATIONFAILURE} ${ENABLE_TFVALIDATE} ${ENABLE_TFFORMAT} ${ENABLE_TFCHECKOV} ${ENABLE_TFSEC}" 47 | # - "cp checkov.xml ${CODEBUILD_SRC_DIR}/reports/checkov.xml" 48 | # - "cp tfsec-junit.xml ${CODEBUILD_SRC_DIR}/reports/tfsec-junit.xml" 49 | artifacts: 50 | files: 51 | - '**/*' 52 | # reports: 53 | # checkov-reports: 54 | # files: 55 | # - "./reports/checkov.xml" 56 | # file-format: "JUNITXML" 57 | # tfsec-reports: 58 | # files: 59 | # - "./reports/tfsec-junit.xml" 60 | # file-format: "JUNITXML" -------------------------------------------------------------------------------- /pipeline/templates/buildspec_validate.yml: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | version: 0.2 8 | 9 | env: 10 | variables: 11 | CODE_SRC_DIR: "." 12 | SCRIPT_DIR: "./templates/scripts" 13 | TF_VERSION: "1.10.0" 14 | ENVIRONMENT: "dev" 15 | SKIPVALIDATIONFAILURE: "Y" 16 | ENABLE_TFVALIDATE: "Y" 17 | ENABLE_TFFORMAT: "Y" 18 | ENABLE_TFCHECKOV: "Y" 19 | ENABLE_TFSEC: "Y" 20 | TFSEC_VERSION: "v1.28.1" 21 | 22 | phases: 23 | install: 24 | runtime-versions: 25 | python: 3.9 26 | golang: 1.14 27 | commands: 28 | - "curl -s -qL -o terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip" 29 | - "unzip -o terraform.zip" 30 | - "mv terraform /bin" 31 | - "rm terraform.zip" 32 | - "yum update -y" 33 | - "yum install -y curl jq awscli" 34 | - "pip3 install checkov" 35 | - "yum install -y golang" 36 | - "wget -q -O tfsec https://github.com/aquasecurity/tfsec/releases/download/${TFSEC_VERSION}/tfsec-linux-amd64" 37 | - "chmod +x ./tfsec" 38 | - "mv ./tfsec /usr/local/bin/tfsec" 39 | build: 40 | commands: 41 | - "cd ${CODEBUILD_SRC_DIR}/${CODE_SRC_DIR}" 42 | - "echo ## TERRAFORM INIT : Initialize the Terraform Workspace" 43 | - "terraform init" 44 | # - "echo ## VALIDATION : Starting ..." 45 | # - "mkdir -p ${CODEBUILD_SRC_DIR}/reports" 46 | # - "/bin/bash ${CODEBUILD_SRC_DIR}/${SCRIPT_DIR}/tf_ssp_validation.sh ${SKIPVALIDATIONFAILURE} ${ENABLE_TFVALIDATE} ${ENABLE_TFFORMAT} ${ENABLE_TFCHECKOV} ${ENABLE_TFSEC}" 47 | # - "cp checkov.xml ${CODEBUILD_SRC_DIR}/reports/checkov.xml" 48 | # - "cp tfsec-junit.xml ${CODEBUILD_SRC_DIR}/reports/tfsec-junit.xml" 49 | artifacts: 50 | files: 51 | - '**/*' 52 | # reports: 53 | # checkov-reports: 54 | # files: 55 | # - "./reports/checkov.xml" 56 | # file-format: "JUNITXML" 57 | # tfsec-reports: 58 | # files: 59 | # - "./reports/tfsec-junit.xml" 60 | # file-format: "JUNITXML" -------------------------------------------------------------------------------- /templates/scripts/tf_ssp_validation.sh: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | #!/bin/bash 8 | 9 | # Accept Command Line Arguments 10 | SKIPVALIDATIONFAILURE=$1 11 | tfValidate=$2 12 | tfFormat=$3 13 | tfCheckov=$4 14 | tfTfsec=$5 15 | # ----------------------------- 16 | 17 | echo "### VALIDATION Overview ###" 18 | echo "-------------------------" 19 | echo "Skip Validation Errors on Failure : ${SKIPVALIDATIONFAILURE}" 20 | echo "Terraform Validate : ${tfValidate}" 21 | echo "Terraform Format : ${tfFormat}" 22 | echo "Terraform checkov : ${tfCheckov}" 23 | echo "Terraform tfsec : ${tfTfsec}" 24 | echo "------------------------" 25 | terraform init 26 | if (( ${tfValidate} == "Y")) 27 | then 28 | echo "## VALIDATION : Validating Terraform code ..." 29 | terraform validate 30 | fi 31 | tfValidateOutput=$? 32 | 33 | if (( ${tfFormat} == "Y")) 34 | then 35 | echo "## VALIDATION : Formatting Terraform code ..." 36 | terraform fmt -recursive 37 | fi 38 | tfFormatOutput=$? 39 | 40 | if (( ${tfCheckov} == "Y")) 41 | then 42 | echo "## VALIDATION : Running checkov ..." 43 | #checkov -s -d . 44 | checkov -o junitxml --framework terraform -d ./ >checkov.xml 45 | fi 46 | tfCheckovOutput=$? 47 | 48 | if (( ${tfTfsec} == "Y")) 49 | then 50 | echo "## VALIDATION : Running tfsec ..." 51 | #tfsec . 52 | tfsec ./ --format junit --out tfsec-junit.xml 53 | fi 54 | tfTfsecOutput=$? 55 | 56 | echo "## VALIDATION Summary ##" 57 | echo "------------------------" 58 | echo "Terraform Validate : ${tfValidateOutput}" 59 | echo "Terraform Format : ${tfFormatOutput}" 60 | echo "Terraform checkov : ${tfCheckovOutput}" 61 | echo "Terraform tfsec : ${tfTfsecOutput}" 62 | echo "------------------------" 63 | 64 | if (( ${SKIPVALIDATIONFAILURE} == "Y" )) 65 | then 66 | #if SKIPVALIDATIONFAILURE is set as Y, then validation failures are skipped during execution 67 | echo "## VALIDATION : Skipping validation failure checks..." 68 | elif (( $tfValidateOutput == 0 && $tfFormatOutput == 0 && $tfCheckovOutput == 0 && $tfTfsecOutput == 0 )) 69 | then 70 | echo "## VALIDATION : Checks Passed!!!" 71 | else 72 | # When validation checks fails, build process is halted. 73 | echo "## ERROR : Validation Failed" 74 | exit 1; 75 | fi -------------------------------------------------------------------------------- /pipeline/templates/scripts/tf_ssp_validation.sh: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | #!/bin/bash 8 | 9 | # Accept Command Line Arguments 10 | SKIPVALIDATIONFAILURE=$1 11 | tfValidate=$2 12 | tfFormat=$3 13 | tfCheckov=$4 14 | tfTfsec=$5 15 | # ----------------------------- 16 | 17 | echo "### VALIDATION Overview ###" 18 | echo "-------------------------" 19 | echo "Skip Validation Errors on Failure : ${SKIPVALIDATIONFAILURE}" 20 | echo "Terraform Validate : ${tfValidate}" 21 | echo "Terraform Format : ${tfFormat}" 22 | echo "Terraform checkov : ${tfCheckov}" 23 | echo "Terraform tfsec : ${tfTfsec}" 24 | echo "------------------------" 25 | terraform init 26 | if (( ${tfValidate} == "Y")) 27 | then 28 | echo "## VALIDATION : Validating Terraform code ..." 29 | terraform validate 30 | fi 31 | tfValidateOutput=$? 32 | 33 | if (( ${tfFormat} == "Y")) 34 | then 35 | echo "## VALIDATION : Formatting Terraform code ..." 36 | terraform fmt -recursive 37 | fi 38 | tfFormatOutput=$? 39 | 40 | if (( ${tfCheckov} == "Y")) 41 | then 42 | echo "## VALIDATION : Running checkov ..." 43 | #checkov -s -d . 44 | checkov -o junitxml --framework terraform -d ./ >checkov.xml 45 | fi 46 | tfCheckovOutput=$? 47 | 48 | if (( ${tfTfsec} == "Y")) 49 | then 50 | echo "## VALIDATION : Running tfsec ..." 51 | #tfsec . 52 | tfsec ./ --format junit --out tfsec-junit.xml 53 | fi 54 | tfTfsecOutput=$? 55 | 56 | echo "## VALIDATION Summary ##" 57 | echo "------------------------" 58 | echo "Terraform Validate : ${tfValidateOutput}" 59 | echo "Terraform Format : ${tfFormatOutput}" 60 | echo "Terraform checkov : ${tfCheckovOutput}" 61 | echo "Terraform tfsec : ${tfTfsecOutput}" 62 | echo "------------------------" 63 | 64 | if (( ${SKIPVALIDATIONFAILURE} == "Y" )) 65 | then 66 | #if SKIPVALIDATIONFAILURE is set as Y, then validation failures are skipped during execution 67 | echo "## VALIDATION : Skipping validation failure checks..." 68 | elif (( $tfValidateOutput == 0 && $tfFormatOutput == 0 && $tfCheckovOutput == 0 && $tfTfsecOutput == 0 )) 69 | then 70 | echo "## VALIDATION : Checks Passed!!!" 71 | else 72 | # When validation checks fails, build process is halted. 73 | echo "## ERROR : Validation Failed" 74 | exit 1; 75 | fi -------------------------------------------------------------------------------- /pipeline/modules/kms/main.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | locals { 8 | account_id = data.aws_caller_identity.current.account_id 9 | } 10 | 11 | resource "aws_kms_key" "encryption_key" { 12 | description = "This key is used to encrypt bucket objects" 13 | deletion_window_in_days = 10 14 | policy = data.aws_iam_policy_document.kms_key_policy_doc.json 15 | enable_key_rotation = true 16 | tags = var.tags 17 | } 18 | 19 | data "aws_iam_policy_document" "kms_key_policy_doc" { 20 | statement { 21 | sid = "Enable IAM User Permissions" 22 | effect = "Allow" 23 | actions = ["kms:*"] 24 | #checkov:skip=CKV_AWS_111:Without this statement, KMS key cannot be managed by root 25 | #checkov:skip=CKV_AWS_109:Without this statement, KMS key cannot be managed by root 26 | resources = ["*"] 27 | 28 | principals { 29 | type = "AWS" 30 | identifiers = ["arn:aws:iam::${local.account_id}:root"] 31 | } 32 | } 33 | 34 | statement { 35 | sid = "Allow access for Key Administrators" 36 | effect = "Allow" 37 | actions = ["kms:*"] 38 | resources = ["*"] 39 | 40 | principals { 41 | type = "AWS" 42 | identifiers = [ 43 | var.codepipeline_role_arn 44 | ] 45 | } 46 | } 47 | 48 | statement { 49 | sid = "Allow use of the key" 50 | effect = "Allow" 51 | actions = [ 52 | "kms:Encrypt", 53 | "kms:Decrypt", 54 | "kms:ReEncrypt*", 55 | "kms:GenerateDataKey*", 56 | "kms:DescribeKey" 57 | ] 58 | resources = ["*"] 59 | 60 | principals { 61 | type = "AWS" 62 | identifiers = [ 63 | var.codepipeline_role_arn 64 | ] 65 | } 66 | } 67 | 68 | statement { 69 | sid = "Allow attachment of persistent resources" 70 | effect = "Allow" 71 | actions = [ 72 | "kms:CreateGrant", 73 | "kms:ListGrants", 74 | "kms:RevokeGrant" 75 | ] 76 | resources = ["*"] 77 | 78 | principals { 79 | type = "AWS" 80 | identifiers = [ 81 | var.codepipeline_role_arn 82 | ] 83 | } 84 | 85 | condition { 86 | test = "Bool" 87 | variable = "kms:GrantIsForAWSResource" 88 | values = ["true"] 89 | } 90 | } 91 | } 92 | -------------------------------------------------------------------------------- /pipeline/variables.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | variable "project_name" { 8 | description = "Unique name for this project" 9 | type = string 10 | } 11 | 12 | 13 | variable "create_new_role" { 14 | description = "Whether to create a new IAM Role. Values are true or false. Defaulted to true always." 15 | type = bool 16 | default = true 17 | } 18 | 19 | variable "codepipeline_iam_role_name" { 20 | description = "Name of the IAM role to be used by the Codepipeline" 21 | type = string 22 | default = "codepipeline-role" 23 | } 24 | 25 | variable "source_repo_name" { 26 | description = "Source repo name of the repository" 27 | type = string 28 | } 29 | 30 | variable "source_repo_branch" { 31 | description = "Default branch in the Source repo for which CodePipeline needs to be configured" 32 | type = string 33 | } 34 | 35 | variable "environment" { 36 | description = "Environment in which the script is run. Eg: dev, prod, etc" 37 | type = string 38 | } 39 | 40 | variable "stage_input" { 41 | description = "Tags to be attached to the CodePipeline" 42 | type = list(map(any)) 43 | } 44 | 45 | variable "build_projects" { 46 | description = "Tags to be attached to the CodePipeline" 47 | type = list(string) 48 | } 49 | 50 | variable "builder_compute_type" { 51 | description = "Relative path to the Apply and Destroy build spec file" 52 | type = string 53 | default = "BUILD_GENERAL1_SMALL" 54 | } 55 | 56 | variable "builder_image" { 57 | description = "Docker Image to be used by codebuild" 58 | type = string 59 | default = "aws/codebuild/amazonlinux2-x86_64-standard:3.0" 60 | } 61 | 62 | variable "builder_type" { 63 | description = "Type of codebuild run environment" 64 | type = string 65 | default = "LINUX_CONTAINER" 66 | } 67 | 68 | variable "builder_image_pull_credentials_type" { 69 | description = "Image pull credentials type used by codebuild project" 70 | type = string 71 | default = "CODEBUILD" 72 | } 73 | 74 | variable "build_project_source" { 75 | description = "aws/codebuild/standard:4.0" 76 | type = string 77 | default = "CODEPIPELINE" 78 | } 79 | 80 | variable "provider_type" { 81 | type = string 82 | description = "Name of the external provider you are connecting to" 83 | } -------------------------------------------------------------------------------- /pipeline/modules/codebuild/README.md: -------------------------------------------------------------------------------- 1 | 2 | ## Requirements 3 | 4 | No requirements. 5 | 6 | ## Providers 7 | 8 | | Name | Version | 9 | |------|---------| 10 | | [aws](#provider\_aws) | n/a | 11 | 12 | ## Modules 13 | 14 | No modules. 15 | 16 | ## Resources 17 | 18 | | Name | Type | 19 | |------|------| 20 | | [aws_codebuild_project.terraform_codebuild_project](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project) | resource | 21 | 22 | ## Inputs 23 | 24 | | Name | Description | Type | Default | Required | 25 | |------|-------------|------|---------|:--------:| 26 | | [build\_project\_source](#input\_build\_project\_source) | Information about the build output artifact location | `string` | n/a | yes | 27 | | [build\_projects](#input\_build\_projects) | List of Names of the CodeBuild projects to be created | `list(string)` | n/a | yes | 28 | | [builder\_compute\_type](#input\_builder\_compute\_type) | Information about the compute resources the build project will use | `string` | n/a | yes | 29 | | [builder\_image](#input\_builder\_image) | Docker image to use for the build project | `string` | n/a | yes | 30 | | [builder\_image\_pull\_credentials\_type](#input\_builder\_image\_pull\_credentials\_type) | Type of credentials AWS CodeBuild uses to pull images in your build. | `string` | n/a | yes | 31 | | [builder\_type](#input\_builder\_type) | Type of build environment to use for related builds | `string` | n/a | yes | 32 | | [kms\_key\_arn](#input\_kms\_key\_arn) | ARN of KMS key for encryption | `string` | n/a | yes | 33 | | [project\_name](#input\_project\_name) | Unique name for this project | `string` | n/a | yes | 34 | | [role\_arn](#input\_role\_arn) | Codepipeline IAM role arn. | `string` | `""` | no | 35 | | [s3\_bucket\_name](#input\_s3\_bucket\_name) | Name of the S3 bucket used to store the deployment artifacts | `string` | n/a | yes | 36 | | [tags](#input\_tags) | Tags to be applied to the codebuild project | `map(any)` | n/a | yes | 37 | 38 | ## Outputs 39 | 40 | | Name | Description | 41 | |------|-------------| 42 | | [arn](#output\_arn) | List of ARNs of the CodeBuild projects | 43 | | [id](#output\_id) | List of IDs of the CodeBuild projects | 44 | | [name](#output\_name) | List of Names of the CodeBuild projects | 45 | -------------------------------------------------------------------------------- /pipeline/modules/repo/README.md: -------------------------------------------------------------------------------- 1 | 2 | ## Requirements 3 | 4 | No requirements. 5 | 6 | ## Providers 7 | 8 | | Name | Version | 9 | |------|---------| 10 | | [aws](#provider\_aws) | n/a | 11 | 12 | ## Modules 13 | 14 | No modules. 15 | 16 | ## Resources 17 | 18 | | Name | Type | 19 | |------|------| 20 | | [aws_codecommit_approval_rule_template.source_repository_approval](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codecommit_approval_rule_template) | resource | 21 | | [aws_codecommit_approval_rule_template_association.source_repository_approval_association](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codecommit_approval_rule_template_association) | resource | 22 | | [aws_codecommit_repository.source_repository](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codecommit_repository) | resource | 23 | | [aws_codecommit_repository.existing_repository](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/codecommit_repository) | data source | 24 | 25 | ## Inputs 26 | 27 | | Name | Description | Type | Default | Required | 28 | |------|-------------|------|---------|:--------:| 29 | | [create\_new\_repo](#input\_create\_new\_repo) | Flag for deciding if a new repository needs to be created | `bool` | `false` | no | 30 | | [kms\_key\_arn](#input\_kms\_key\_arn) | Name of the project to be prefixed to create the s3 bucket | `string` | n/a | yes | 31 | | [repo\_approvers\_arn](#input\_repo\_approvers\_arn) | ARN or ARN pattern for the IAM User/Role/Group etc that can be used for approving Pull Requests | `string` | n/a | yes | 32 | | [source\_repository\_branch](#input\_source\_repository\_branch) | Branch of the Source CodeCommit repository used in pipeline | `string` | n/a | yes | 33 | | [source\_repository\_name](#input\_source\_repository\_name) | Name of the Source CodeCommit repository used by the pipeline | `string` | n/a | yes | 34 | | [tags](#input\_tags) | Tags to be attached to the source CodeCommit repository | `map(any)` | n/a | yes | 35 | 36 | ## Outputs 37 | 38 | | Name | Description | 39 | |------|-------------| 40 | | [arn](#output\_arn) | LList containing the arn of the CodeCommit repositories | 41 | | [clone\_url\_http](#output\_clone\_url\_http) | List containing the clone url of the CodeCommit repositories | 42 | | [repository\_name](#output\_repository\_name) | List containing the name of the CodeCommit repositories | 43 | -------------------------------------------------------------------------------- /pipeline/modules/iam-role/README.md: -------------------------------------------------------------------------------- 1 | 2 | ## Requirements 3 | 4 | No requirements. 5 | 6 | ## Providers 7 | 8 | | Name | Version | 9 | |------|---------| 10 | | [aws](#provider\_aws) | n/a | 11 | 12 | ## Modules 13 | 14 | No modules. 15 | 16 | ## Resources 17 | 18 | | Name | Type | 19 | |------|------| 20 | | [aws_accessanalyzer_analyzer.codepipeline_analyzer](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/accessanalyzer_analyzer) | resource | 21 | | [aws_iam_policy.codepipeline_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource | 22 | | [aws_iam_role.codepipeline_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | 23 | | [aws_iam_role_policy_attachment.codepipeline_role_attach](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | 24 | | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | 25 | | [aws_iam_role.existing_codepipeline_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_role) | data source | 26 | | [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source | 27 | | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | 28 | 29 | ## Inputs 30 | 31 | | Name | Description | Type | Default | Required | 32 | |------|-------------|------|---------|:--------:| 33 | | [codepipeline\_iam\_role\_name](#input\_codepipeline\_iam\_role\_name) | Name of the IAM role to be used by the project | `string` | n/a | yes | 34 | | [create\_new\_role](#input\_create\_new\_role) | Flag for deciding if a new role needs to be created | `bool` | `true` | no | 35 | | [kms\_key\_arn](#input\_kms\_key\_arn) | ARN of KMS key for encryption | `string` | n/a | yes | 36 | | [project\_name](#input\_project\_name) | Unique name for this project | `string` | n/a | yes | 37 | | [s3\_bucket\_arn](#input\_s3\_bucket\_arn) | The ARN of the S3 Bucket | `string` | n/a | yes | 38 | | [source\_repository\_name](#input\_source\_repository\_name) | Name of the Source CodeCommit repository | `string` | n/a | yes | 39 | | [tags](#input\_tags) | Tags to be attached to the IAM Role | `map(any)` | n/a | yes | 40 | 41 | ## Outputs 42 | 43 | | Name | Description | 44 | |------|-------------| 45 | | [role\_arn](#output\_role\_arn) | The ARN of the IAM Role | 46 | | [role\_name](#output\_role\_name) | The ARN of the IAM Role | 47 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *main* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | -------------------------------------------------------------------------------- /main.tf: -------------------------------------------------------------------------------- 1 | #S3 2 | module "s3_bucket" { 3 | source = "./modules/s3" 4 | bucket_name = "${var.kb_s3_bucket_name_prefix}-${local.region_short}-${local.account_id}" 5 | } 6 | 7 | #Create bucket for custom model 8 | module "custom_model_bucket" { 9 | source = "./modules/s3" 10 | bucket_name = "sample-model-training-${local.region_short}-${local.account_id}" 11 | } 12 | 13 | # Knowledge base resource role 14 | module "sample_iam" { 15 | source = "./modules/iam" 16 | action_group_name = var.action_group_name 17 | agent_name = var.agent_name 18 | kb_name = var.kb_name 19 | s3_arn = module.s3_bucket.arn 20 | kb_model_arn = data.aws_bedrock_foundation_model.kb.model_arn 21 | lambda_iam_policy = data.aws_iam_policy.lambda_basic_execution.arn 22 | custom_model_s3_arn = module.custom_model_bucket.arn 23 | } 24 | 25 | module "custom-model-generation" { 26 | source = "./modules/custom-model" 27 | custom_model_bucket = module.custom_model_bucket.bucket_name 28 | bedrock_custom_role_arn = module.sample_iam.bedrock_custom_role_arn 29 | } 30 | 31 | module "action_group_lambda" { 32 | source = "./modules/lambda" 33 | action_group_name = var.action_group_name 34 | lambda_role_arn = module.sample_iam.lambda_role_arn 35 | subnet_ids = var.subnet_ids 36 | security_group_ids = var.security_group_ids 37 | } 38 | 39 | module "opensearch" { 40 | source = "./modules/opensearch" 41 | kb_oss_collection_name = var.kb_oss_collection_name 42 | bedrock_role_arn = module.sample_iam.bedrock_role_arn 43 | index_name = "bedrock-knowledge-base-default-index" 44 | } 45 | 46 | resource "time_sleep" "delay" { 47 | depends_on = [module.opensearch.opensearch_index_name] 48 | create_duration = "60s" 49 | } 50 | 51 | module "knowledge-base" { 52 | source = "./modules/knowledge-base" 53 | kb_name = var.kb_name 54 | bedrock_role_arn = module.sample_iam.bedrock_role_arn 55 | bedrock_role_name = module.sample_iam.bedrock_role_name 56 | kb_model_arn = data.aws_bedrock_foundation_model.kb.model_arn 57 | opensearch_arn = module.opensearch.opensearch_collection_arn 58 | s3_arn = module.s3_bucket.arn 59 | opensearch_index_name = module.opensearch.opensearch_index_name 60 | depends_on = [time_sleep.delay] 61 | } 62 | 63 | module "guardrails" { 64 | source = "./modules/guardrails" 65 | name = "sample_guardrail" 66 | } 67 | 68 | module "bedrock-agent" { 69 | source = "./modules/bedrock-agent" 70 | agent_name = var.agent_name 71 | agent_model_id = var.agent_model_id 72 | kb_id = module.knowledge-base.kb_id 73 | guardrail_identifier = module.guardrails.guardrail_identifier 74 | guardrail_version = module.guardrails.guardrail_version 75 | action_group_lambda_arn = module.action_group_lambda.lambda_arn 76 | kb_arn = module.knowledge-base.kb_arn 77 | } 78 | 79 | resource "terraform_data" "sample_asst_prepare" { 80 | triggers_replace = { 81 | sample_kb_state = sha256(jsonencode(module.knowledge-base)) 82 | } 83 | provisioner "local-exec" { 84 | command = "aws bedrock-agent prepare-agent --agent-id ${module.bedrock-agent.agent_id}" 85 | } 86 | } -------------------------------------------------------------------------------- /modules/bedrock-agent/main.tf: -------------------------------------------------------------------------------- 1 | resource "awscc_bedrock_agent" "sample_asst" { 2 | agent_name = var.agent_name 3 | description = "sample agent configuration" 4 | agent_resource_role_arn = aws_iam_role.bedrock_agent_sample_asst.arn 5 | foundation_model = data.aws_bedrock_foundation_model.agent.model_id 6 | instruction = file("${path.module}/instruction.txt") 7 | knowledge_bases = [{ 8 | description = "sample knowledge base" 9 | knowledge_base_id = var.kb_id 10 | knowledge_base_state = "ENABLED" 11 | }] 12 | guardrail_configuration = { 13 | guardrail_identifier = var.guardrail_identifier 14 | guardrail_version = var.guardrail_version 15 | } 16 | idle_session_ttl_in_seconds = 600 17 | auto_prepare = true 18 | action_groups = [{ 19 | action_group_name = var.action_group_name 20 | description = var.action_group_desc 21 | function_schema = { 22 | functions = [{ 23 | name = "sample-function" 24 | description = "sample function" 25 | parameters = { 26 | param1 = { 27 | type = "string" 28 | description = "The first parameter" 29 | required = true 30 | } 31 | param2 = { 32 | type = "integer" 33 | description = "The second parameter" 34 | required = false 35 | } 36 | } 37 | return_value = { 38 | type = "string" 39 | description = "The return value" 40 | } 41 | }] 42 | } 43 | action_group_executor = { 44 | lambda = var.action_group_lambda_arn 45 | } 46 | }] 47 | 48 | tags = { 49 | "Modified By" = "AWSCC" 50 | } 51 | 52 | } 53 | 54 | # Agent resource role 55 | resource "aws_iam_role" "bedrock_agent_sample_asst" { 56 | name = "AmazonBedrockExecutionRoleForAgents_${var.agent_name}" 57 | assume_role_policy = jsonencode({ 58 | Version = "2012-10-17" 59 | Statement = [ 60 | { 61 | Action = "sts:AssumeRole" 62 | Effect = "Allow" 63 | Principal = { 64 | Service = "bedrock.amazonaws.com" 65 | } 66 | Condition = { 67 | StringEquals = { 68 | "aws:SourceAccount" = local.account_id 69 | } 70 | ArnLike = { 71 | "aws:SourceArn" = "arn:${local.partition}:bedrock:${local.region}:${local.account_id}:agent/*" 72 | } 73 | } 74 | } 75 | ] 76 | }) 77 | } 78 | 79 | resource "aws_iam_role_policy" "bedrock_agent_sample_asst_model" { 80 | name = "AmazonBedrockAgentBedrockFoundationModelPolicy_${var.agent_name}" 81 | role = aws_iam_role.bedrock_agent_sample_asst.name 82 | policy = jsonencode({ 83 | Version = "2012-10-17" 84 | Statement = [ 85 | { 86 | Action = "bedrock:InvokeModel" 87 | Effect = "Allow" 88 | Resource = data.aws_bedrock_foundation_model.agent.model_arn 89 | } 90 | ] 91 | }) 92 | } 93 | 94 | resource "aws_iam_role_policy" "bedrock_agent_sample_asst_kb" { 95 | name = "AmazonBedrockAgentBedrockKnowledgeBasePolicy_${var.agent_name}" 96 | role = aws_iam_role.bedrock_agent_sample_asst.name 97 | policy = jsonencode({ 98 | Version = "2012-10-17" 99 | Statement = [ 100 | { 101 | Action = "bedrock:Retrieve" 102 | Effect = "Allow" 103 | Resource = var.kb_arn 104 | } 105 | ] 106 | }) 107 | } -------------------------------------------------------------------------------- /modules/iam/main.tf: -------------------------------------------------------------------------------- 1 | resource "aws_iam_role" "bedrock_kb_sample_kb" { 2 | name = "AmazonBedrockExecutionRoleForKnowledgeBase_${var.kb_name}" 3 | assume_role_policy = jsonencode({ 4 | Version = "2012-10-17" 5 | Statement = [ 6 | { 7 | Action = "sts:AssumeRole" 8 | Effect = "Allow" 9 | Principal = { 10 | Service = "bedrock.amazonaws.com" 11 | } 12 | Condition = { 13 | StringEquals = { 14 | "aws:SourceAccount" = local.account_id 15 | } 16 | ArnLike = { 17 | "aws:SourceArn" = "arn:${local.partition}:bedrock:${local.region}:${local.account_id}:knowledge-base/*" 18 | } 19 | } 20 | } 21 | ] 22 | }) 23 | } 24 | 25 | resource "aws_iam_role_policy" "bedrock_kb_sample_kb_model" { 26 | name = "AmazonBedrockFoundationModelPolicyForKnowledgeBase_${var.kb_name}" 27 | role = aws_iam_role.bedrock_kb_sample_kb.name 28 | policy = jsonencode({ 29 | Version = "2012-10-17" 30 | Statement = [ 31 | { 32 | Action = "bedrock:InvokeModel" 33 | Effect = "Allow" 34 | Resource = var.kb_model_arn 35 | } 36 | ] 37 | }) 38 | } 39 | 40 | resource "aws_iam_role_policy" "bedrock_kb_sample_kb_s3" { 41 | name = "AmazonBedrockS3PolicyForKnowledgeBase_${var.kb_name}" 42 | role = aws_iam_role.bedrock_kb_sample_kb.name 43 | policy = jsonencode({ 44 | Version = "2012-10-17" 45 | Statement = [ 46 | { 47 | Sid = "S3ListGetPutObjectStatement" 48 | Action = ["s3:List*", "s3:Get*", "s3:PutObject"] 49 | Effect = "Allow" 50 | Resource = [var.s3_arn, var.custom_model_s3_arn, "${var.s3_arn}/*", "${var.custom_model_s3_arn}/*"] 51 | Condition = { 52 | StringEquals = { 53 | "aws:PrincipalAccount" = local.account_id 54 | } 55 | } }, 56 | { 57 | Sid = "KMSPermissions" 58 | Action = ["kms:*"] 59 | Effect = "Allow" 60 | Resource = ["*"] 61 | } 62 | ] 63 | }) 64 | } 65 | 66 | 67 | #Action group Lambda execution role 68 | resource "aws_iam_role" "lambda_sample_api" { 69 | name = "FunctionExecutionRoleForLambda_${var.action_group_name}" 70 | assume_role_policy = jsonencode({ 71 | Version = "2012-10-17" 72 | Statement = [ 73 | { 74 | Action = "sts:AssumeRole" 75 | Effect = "Allow" 76 | Principal = { 77 | Service = "lambda.amazonaws.com" 78 | } 79 | Condition = { 80 | StringEquals = { 81 | "aws:SourceAccount" = local.account_id 82 | } 83 | } 84 | } 85 | ] 86 | }) 87 | managed_policy_arns = [var.lambda_iam_policy] 88 | } 89 | 90 | #custom model policy 91 | data "aws_iam_policy_document" "assume_role_policy" { 92 | statement { 93 | actions = ["sts:AssumeRole"] 94 | effect = "Allow" 95 | principals { 96 | type = "Service" 97 | identifiers = ["bedrock.amazonaws.com"] 98 | } 99 | condition { 100 | test = "StringEquals" 101 | variable = "aws:SourceAccount" 102 | values = [data.aws_caller_identity.this.account_id] 103 | } 104 | condition { 105 | test = "ArnEquals" 106 | variable = "aws:SourceArn" 107 | values = ["arn:${local.partition}:bedrock:${local.region}:${local.account_id}:model-customization-job/*"] 108 | } 109 | } 110 | } 111 | 112 | resource "aws_iam_policy" "bedrock_custom_policy" { 113 | name_prefix = "BedrockCM-" 114 | description = "Policy for Bedrock Custom Models customization jobs" 115 | policy = data.aws_iam_policy_document.bedrock_custom_policy.json 116 | } 117 | 118 | resource "aws_iam_role" "bedrock_custom_role" { 119 | name_prefix = "BedrockCM-" 120 | description = "Role for Bedrock Custom Models customization jobs" 121 | assume_role_policy = data.aws_iam_policy_document.assume_role_policy.json 122 | managed_policy_arns = [aws_iam_policy.bedrock_custom_policy.arn] 123 | } -------------------------------------------------------------------------------- /pipeline/main.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | #Module for creating a new S3 bucket for storing pipeline artifacts 8 | module "s3_artifacts_bucket" { 9 | source = "./modules/s3" 10 | project_name = var.project_name 11 | kms_key_arn = module.codepipeline_kms.arn 12 | codepipeline_role_arn = module.codepipeline_iam_role.role_arn 13 | tags = { 14 | Project_Name = var.project_name 15 | Environment = var.environment 16 | Account_ID = local.account_id 17 | Region = local.region 18 | } 19 | } 20 | 21 | # Resources 22 | 23 | # Module for Infrastructure Source code repository 24 | module "source_repo" { 25 | source = "./modules/repo" 26 | provider_type = var.provider_type 27 | } 28 | 29 | # Module for Infrastructure Validation - CodeBuild 30 | module "codebuild_terraform" { 31 | depends_on = [ 32 | module.source_repo 33 | ] 34 | source = "./modules/codebuild" 35 | 36 | project_name = var.project_name 37 | role_arn = module.codepipeline_iam_role.role_arn 38 | s3_bucket_name = module.s3_artifacts_bucket.bucket 39 | build_projects = var.build_projects 40 | build_project_source = var.build_project_source 41 | builder_compute_type = var.builder_compute_type 42 | builder_image = var.builder_image 43 | builder_image_pull_credentials_type = var.builder_image_pull_credentials_type 44 | builder_type = var.builder_type 45 | kms_key_arn = module.codepipeline_kms.arn 46 | tags = { 47 | Project_Name = var.project_name 48 | Environment = var.environment 49 | Account_ID = local.account_id 50 | Region = local.region 51 | } 52 | } 53 | 54 | module "codepipeline_kms" { 55 | source = "./modules/kms" 56 | codepipeline_role_arn = module.codepipeline_iam_role.role_arn 57 | tags = { 58 | Project_Name = var.project_name 59 | Environment = var.environment 60 | Account_ID = local.account_id 61 | Region = local.region 62 | } 63 | 64 | } 65 | 66 | module "codepipeline_iam_role" { 67 | source = "./modules/iam-role" 68 | project_name = var.project_name 69 | create_new_role = var.create_new_role 70 | codepipeline_iam_role_name = var.create_new_role == true ? "${var.project_name}-codepipeline-role" : var.codepipeline_iam_role_name 71 | source_repository_name = var.source_repo_name 72 | kms_key_arn = module.codepipeline_kms.arn 73 | s3_bucket_arn = module.s3_artifacts_bucket.arn 74 | tags = { 75 | Project_Name = var.project_name 76 | Environment = var.environment 77 | Account_ID = local.account_id 78 | Region = local.region 79 | } 80 | } 81 | # Module for Infrastructure Validate, Plan, Apply and Destroy - CodePipeline 82 | module "codepipeline_terraform" { 83 | depends_on = [ 84 | module.codebuild_terraform, 85 | module.s3_artifacts_bucket 86 | ] 87 | source = "./modules/codepipeline" 88 | 89 | project_name = var.project_name 90 | source_repo_name = var.source_repo_name 91 | source_repo_branch = var.source_repo_branch 92 | s3_bucket_name = module.s3_artifacts_bucket.bucket 93 | codepipeline_role_arn = module.codepipeline_iam_role.role_arn 94 | stages = var.stage_input 95 | kms_key_arn = module.codepipeline_kms.arn 96 | connection_arn = module.source_repo.connection_arn 97 | tags = { 98 | Project_Name = var.project_name 99 | Environment = var.environment 100 | Account_ID = local.account_id 101 | Region = local.region 102 | } 103 | } 104 | -------------------------------------------------------------------------------- /modules/opensearch/main.tf: -------------------------------------------------------------------------------- 1 | resource "aws_opensearchserverless_access_policy" "sample_kb" { 2 | name = var.kb_oss_collection_name 3 | type = "data" 4 | policy = jsonencode([ 5 | { 6 | Rules = [ 7 | { 8 | ResourceType = "index" 9 | Resource = [ 10 | "index/${var.kb_oss_collection_name}/*" 11 | ] 12 | Permission = [ 13 | "aoss:CreateIndex", 14 | "aoss:DeleteIndex", # Required for Terraform 15 | "aoss:DescribeIndex", 16 | "aoss:ReadDocument", 17 | "aoss:UpdateIndex", 18 | "aoss:WriteDocument" 19 | ] 20 | }, 21 | { 22 | ResourceType = "collection" 23 | Resource = [ 24 | "collection/${var.kb_oss_collection_name}" 25 | ] 26 | Permission = [ 27 | "aoss:CreateCollectionItems", 28 | "aoss:DescribeCollectionItems", 29 | "aoss:UpdateCollectionItems" 30 | ] 31 | } 32 | ], 33 | Principal = [ 34 | var.bedrock_role_arn, 35 | data.aws_caller_identity.this.arn, 36 | "arn:aws:sts::${data.aws_caller_identity.this.account_id}:assumed-role/bedrock-codepipeline-role/*" 37 | ] 38 | } 39 | ]) 40 | } 41 | 42 | 43 | resource "aws_opensearchserverless_security_policy" "sample_kb_encryption" { 44 | name = var.kb_oss_collection_name 45 | type = "encryption" 46 | policy = jsonencode({ 47 | Rules = [ 48 | { 49 | Resource = [ 50 | "collection/${var.kb_oss_collection_name}" 51 | ] 52 | ResourceType = "collection" 53 | } 54 | ], 55 | AWSOwnedKey = true 56 | }) 57 | } 58 | 59 | resource "aws_opensearchserverless_security_policy" "sample_kb_network" { 60 | name = var.kb_oss_collection_name 61 | type = "network" 62 | policy = jsonencode([ 63 | { 64 | Rules = [ 65 | { 66 | ResourceType = "collection" 67 | Resource = [ 68 | "collection/${var.kb_oss_collection_name}" 69 | ] 70 | }, 71 | { 72 | ResourceType = "dashboard" 73 | Resource = [ 74 | "collection/${var.kb_oss_collection_name}" 75 | ] 76 | } 77 | ] 78 | AllowFromPublic = true 79 | } 80 | ]) 81 | } 82 | 83 | resource "aws_opensearchserverless_collection" "sample_kb" { 84 | name = var.kb_oss_collection_name 85 | type = "VECTORSEARCH" 86 | depends_on = [ 87 | aws_opensearchserverless_access_policy.sample_kb, 88 | aws_opensearchserverless_security_policy.sample_kb_encryption, 89 | aws_opensearchserverless_security_policy.sample_kb_network 90 | ] 91 | } 92 | 93 | provider "opensearch" { 94 | url = aws_opensearchserverless_collection.sample_kb.collection_endpoint 95 | healthcheck = false 96 | } 97 | 98 | resource "time_sleep" "wait_before_index_creation" { 99 | depends_on = [aws_opensearchserverless_collection.sample_kb] 100 | create_duration = "60s" # Wait for 60 seconds before creating the index 101 | } 102 | 103 | resource "opensearch_index" "sample_kb" { 104 | name = var.index_name 105 | number_of_shards = "2" 106 | number_of_replicas = "0" 107 | index_knn = true 108 | index_knn_algo_param_ef_search = "512" 109 | mappings = <<-EOF 110 | { 111 | "properties": { 112 | "bedrock-knowledge-base-default-vector": { 113 | "type": "knn_vector", 114 | "dimension": 1536, 115 | "method": { 116 | "name": "hnsw", 117 | "engine": "faiss", 118 | "parameters": { 119 | "m": 16, 120 | "ef_construction": 512 121 | }, 122 | "space_type": "l2" 123 | } 124 | }, 125 | "AMAZON_BEDROCK_METADATA": { 126 | "type": "text", 127 | "index": "false" 128 | }, 129 | "AMAZON_BEDROCK_TEXT_CHUNK": { 130 | "type": "text", 131 | "index": "true" 132 | } 133 | } 134 | } 135 | EOF 136 | force_destroy = true 137 | depends_on = [time_sleep.wait_before_index_creation] 138 | } -------------------------------------------------------------------------------- /pipeline/modules/iam-role/main.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | resource "aws_iam_role" "codepipeline_role" { 8 | count = var.create_new_role ? 1 : 0 9 | name = var.codepipeline_iam_role_name 10 | tags = var.tags 11 | assume_role_policy = < 2 | ## Requirements 3 | 4 | No requirements. 5 | 6 | ## Providers 7 | 8 | | Name | Version | 9 | |------|---------| 10 | | [aws](#provider\_aws) | n/a | 11 | | [aws.replication](#provider\_aws.replication) | n/a | 12 | 13 | ## Modules 14 | 15 | No modules. 16 | 17 | ## Resources 18 | 19 | | Name | Type | 20 | |------|------| 21 | | [aws_iam_policy.replication_s3_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource | 22 | | [aws_iam_role.replication_s3_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | 23 | | [aws_iam_role_policy_attachment.replication_s3_role_attach](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | 24 | | [aws_s3_bucket.codepipeline_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource | 25 | | [aws_s3_bucket.replication_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource | 26 | | [aws_s3_bucket_acl.codepipeline_bucket_acl](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource | 27 | | [aws_s3_bucket_acl.replication_bucket_acl](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource | 28 | | [aws_s3_bucket_logging.codepipeline_bucket_logging](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_logging) | resource | 29 | | [aws_s3_bucket_logging.replication_bucket_logging](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_logging) | resource | 30 | | [aws_s3_bucket_policy.bucket_policy_codepipeline_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) | resource | 31 | | [aws_s3_bucket_policy.bucket_policy_replication_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) | resource | 32 | | [aws_s3_bucket_public_access_block.codepipeline_bucket_access](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource | 33 | | [aws_s3_bucket_public_access_block.replication_bucket_access](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource | 34 | | [aws_s3_bucket_replication_configuration.replication_config](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_replication_configuration) | resource | 35 | | [aws_s3_bucket_server_side_encryption_configuration.codepipeline_bucket_encryption](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource | 36 | | [aws_s3_bucket_server_side_encryption_configuration.replication_bucket_encryption](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource | 37 | | [aws_s3_bucket_versioning.codepipeline_bucket_versioning](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) | resource | 38 | | [aws_s3_bucket_versioning.replication_bucket_versioning](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) | resource | 39 | | [aws_iam_policy_document.bucket_policy_doc_codepipeline_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | 40 | | [aws_iam_policy_document.bucket_policy_doc_replication_bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source | 41 | 42 | ## Inputs 43 | 44 | | Name | Description | Type | Default | Required | 45 | |------|-------------|------|---------|:--------:| 46 | | [codepipeline\_role\_arn](#input\_codepipeline\_role\_arn) | ARN of the codepipeline IAM role | `string` | n/a | yes | 47 | | [kms\_key\_arn](#input\_kms\_key\_arn) | ARN of KMS key for encryption | `string` | n/a | yes | 48 | | [project\_name](#input\_project\_name) | Name of the project to be prefixed to create the s3 bucket | `string` | n/a | yes | 49 | | [tags](#input\_tags) | Tags to be associated with the S3 bucket | `map(any)` | n/a | yes | 50 | 51 | ## Outputs 52 | 53 | | Name | Description | 54 | |------|-------------| 55 | | [arn](#output\_arn) | The ARN of the S3 Bucket | 56 | | [bucket](#output\_bucket) | The Name of the S3 Bucket | 57 | | [bucket\_url](#output\_bucket\_url) | The URL of the S3 Bucket | 58 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # Custom Bedrock model deployment 3 | 4 | ![](./images/ML-Model-Deployment-Page-2.drawio.png) 5 | 6 | This repository contains sample code demonstrating how to use Terraform to provision a custom model using Bedrock with a dataset which answers some common banking questions for an existing or prospective customer. 7 | 8 | The root Terraform module configuration provisions the below components into a subnet of your choice with the necessary security group attached. 9 | 10 | * Amazon Bedrock 11 | * Bedrock Custom model training job and a custom model on successful run of the training job. 12 | * Bedrock Guardrail 13 | * Bedrock Knowledge Base 14 | * Bedrock Agents 15 | * Amazon Opensearch Collection and Index 16 | * Amazon S3 17 | * AWS IAM 18 | 19 | > [!IMPORTANT] 20 | > A destroy operation on the stack doesn't delete the custom model training job, but rather stops it. Keep in mind that that the name of the job has to be unique on an account. We are using the random provider to generate a custom name for the new deployments of the stack. 21 | 22 | The solution uses the dataset `SohamNale/Banking_Dataset_for_LLM_Finetuning` and generates training data in the form of a jsonl file for the format below. 23 | 24 | ``` 25 | { 26 | "prompt": "What is a bank account?", 27 | "completion": "A bank account is a deposit account held at a financial institution that allows a customer to store money and perform financial transactions." 28 | } 29 | 30 | ``` 31 | 32 | The training dataset is uploaded to an S3 bucket created as part of the stack and referenced as as the training data for the custome model training job. The job ,when finished fine tuning the foundation model ,creates a custom model which can be used using a provisioned throughput with a no commitment or term based commitment based on your choice. 33 | 34 | ## Prerequisites 35 | 36 | You need to have an AWS account and an AWS Identity and Access Management (IAM) role and user with permissions to create and manage the necessary resources and components for this application. Before proceeding, if you have not previously done so, you must request access to the following Amazon Bedrock models: 37 | 38 | * Amazon: amazon.titan-embed-text-v1 39 | * Anthropic: anthropic.claude-3-haiku-20240307-v1:0 40 | 41 | ## Requirements 42 | 43 | | Name | Version | 44 | |------|---------| 45 | | [terraform](#requirement\_terraform) | ~> 1.0 | 46 | | [aws](#requirement\_aws) | ~> 5.48 | 47 | | [awscc](#requirement\_awscc) | ~> 1.0 | 48 | | [opensearch](#requirement\_opensearch) | = 2.2.0 | 49 | | [random](#requirement\_random) | ~>3.0 | 50 | 51 | ## Providers 52 | 53 | | Name | Version | 54 | |------|---------| 55 | | [aws](#provider\_aws) | ~> 5.48 | 56 | | [terraform](#provider\_terraform) | n/a | 57 | | [time](#provider\_time) | n/a | 58 | 59 | ## Modules 60 | 61 | | Name | Source | Version | 62 | |------|--------|---------| 63 | | [action\_group\_lambda](#module\_action\_group\_lambda) | ./modules/lambda | n/a | 64 | | [bedrock-agent](#module\_bedrock-agent) | ./modules/bedrock-agent | n/a | 65 | | [custom-model-generation](#module\_custom-model-generation) | ./modules/custom-model | n/a | 66 | | [custom\_model\_bucket](#module\_custom\_model\_bucket) | ./modules/s3 | n/a | 67 | | [guardrails](#module\_guardrails) | ./modules/guardrails | n/a | 68 | | [knowledge-base](#module\_knowledge-base) | ./modules/knowledge-base | n/a | 69 | | [opensearch](#module\_opensearch) | ./modules/opensearch | n/a | 70 | | [s3\_bucket](#module\_s3\_bucket) | ./modules/s3 | n/a | 71 | | [sample\_iam](#module\_sample\_iam) | ./modules/iam | n/a | 72 | 73 | ## Resources 74 | 75 | | Name | Type | 76 | |------|------| 77 | | [terraform_data.sample_asst_prepare](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource | 78 | | [time_sleep.delay](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource | 79 | | [aws_bedrock_foundation_model.kb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/bedrock_foundation_model) | data source | 80 | | [aws_caller_identity.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | 81 | | [aws_iam_policy.lambda_basic_execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source | 82 | | [aws_partition.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source | 83 | | [aws_region.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | 84 | 85 | ## Inputs 86 | 87 | | Name | Description | Type | Default | Required | 88 | |------|-------------|------|---------|:--------:| 89 | | [action\_group\_desc](#input\_action\_group\_desc) | The description of the action group. | `string` | `"An action group that provides sample information."` | no | 90 | | [action\_group\_name](#input\_action\_group\_name) | The name of the action group. | `string` | `"sampleAPI"` | no | 91 | | [agent\_desc](#input\_agent\_desc) | The description of the agent. | `string` | `"An assistant that provides sample information."` | no | 92 | | [agent\_model\_id](#input\_agent\_model\_id) | The ID of the foundational model used by the agent. | `string` | `"anthropic.claude-3-haiku-20240307-v1:0"` | no | 93 | | [agent\_name](#input\_agent\_name) | The name of the agent. | `string` | `"sampleAssistant"` | no | 94 | | [kb\_model\_id](#input\_kb\_model\_id) | The ID of the foundational model used by the knowledge base. | `string` | `"amazon.titan-embed-text-v1"` | no | 95 | | [kb\_name](#input\_kb\_name) | The name of the knowledge base. | `string` | `"sample"` | no | 96 | | [kb\_oss\_collection\_name](#input\_kb\_oss\_collection\_name) | The name of the OpenSearch Service (OSS) collection for the knowledge base. | `string` | `"bedrock-sample-kb"` | no | 97 | | [kb\_s3\_bucket\_name\_prefix](#input\_kb\_s3\_bucket\_name\_prefix) | The name prefix of the S3 bucket for the data source of the knowledge base. | `string` | `"sample-kb"` | no | 98 | | [security\_group\_ids](#input\_security\_group\_ids) | The list of security group IDs to be associated with the Lambda function. | `list(string)` | 
[
  "sg-123"
]
| no | 99 | | [subnet\_ids](#input\_subnet\_ids) | The list of subnet IDs where the Lambda function will be deployed. | `list(string)` | 
[
  "subnet-123"
]
| no | 100 | 101 | ## Outputs 102 | 103 | No outputs. 104 | -------------------------------------------------------------------------------- /pipeline/modules/s3/main.tf: -------------------------------------------------------------------------------- 1 | #This solution, non-production-ready template describes AWS Codepipeline based CICD Pipeline for terraform code deployment. 2 | #© 2023 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. 3 | #This AWS Content is provided subject to the terms of the AWS Customer Agreement available at 4 | #http://aws.amazon.com/agreement or other written agreement between Customer and either 5 | #Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both. 6 | 7 | resource "aws_iam_role" "replication_s3_role" { 8 | name = "${var.project_name}-replication-role" 9 | 10 | assume_role_policy = < Settings -> Connections -> Choose the connection created by the terraform apply. 89 | 90 | #### Step 7: Trigger the pipeline created in the Installation step. 91 | 92 | 93 | ## Requirements 94 | 95 | | Name | Version | 96 | |------|---------| 97 | | [terraform](#requirement\_terraform) | ~> 1.0 | 98 | | [aws](#requirement\_aws) | ~> 5.48 | 99 | 100 | ## Providers 101 | 102 | | Name | Version | 103 | |------|---------| 104 | | [aws](#provider\_aws) | ~> 5.48 | 105 | 106 | ## Modules 107 | 108 | | Name | Source | Version | 109 | |------|--------|---------| 110 | | [codebuild\_terraform](#module\_codebuild\_terraform) | ./modules/codebuild | n/a | 111 | | [codepipeline\_iam\_role](#module\_codepipeline\_iam\_role) | ./modules/iam-role | n/a | 112 | | [codepipeline\_kms](#module\_codepipeline\_kms) | ./modules/kms | n/a | 113 | | [codepipeline\_terraform](#module\_codepipeline\_terraform) | ./modules/codepipeline | n/a | 114 | | [s3\_artifacts\_bucket](#module\_s3\_artifacts\_bucket) | ./modules/s3 | n/a | 115 | | [source\_repo](#module\_source\_repo) | ./modules/repo | n/a | 116 | 117 | ## Resources 118 | 119 | | Name | Type | 120 | |------|------| 121 | | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | 122 | | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | 123 | 124 | ## Inputs 125 | 126 | | Name | Description | Type | Default | Required | 127 | |------|-------------|------|---------|:--------:| 128 | | [build\_project\_source](#input\_build\_project\_source) | aws/codebuild/standard:4.0 | `string` | `"CODEPIPELINE"` | no | 129 | | [build\_projects](#input\_build\_projects) | Tags to be attached to the CodePipeline | `list(string)` | n/a | yes | 130 | | [builder\_compute\_type](#input\_builder\_compute\_type) | Relative path to the Apply and Destroy build spec file | `string` | `"BUILD_GENERAL1_SMALL"` | no | 131 | | [builder\_image](#input\_builder\_image) | Docker Image to be used by codebuild | `string` | `"aws/codebuild/amazonlinux2-x86_64-standard:3.0"` | no | 132 | | [builder\_image\_pull\_credentials\_type](#input\_builder\_image\_pull\_credentials\_type) | Image pull credentials type used by codebuild project | `string` | `"CODEBUILD"` | no | 133 | | [builder\_type](#input\_builder\_type) | Type of codebuild run environment | `string` | `"LINUX_CONTAINER"` | no | 134 | | [codepipeline\_iam\_role\_name](#input\_codepipeline\_iam\_role\_name) | Name of the IAM role to be used by the Codepipeline | `string` | `"codepipeline-role"` | no | 135 | | [create\_new\_role](#input\_create\_new\_role) | Whether to create a new IAM Role. Values are true or false. Defaulted to true always. | `bool` | `true` | no | 136 | | [environment](#input\_environment) | Environment in which the script is run. Eg: dev, prod, etc | `string` | n/a | yes | 137 | | [project\_name](#input\_project\_name) | Unique name for this project | `string` | n/a | yes | 138 | | [provider\_type](#input\_provider\_type) | Name of the external provider you are connecting to | `string` | n/a | yes | 139 | | [source\_repo\_branch](#input\_source\_repo\_branch) | Default branch in the Source repo for which CodePipeline needs to be configured | `string` | n/a | yes | 140 | | [source\_repo\_name](#input\_source\_repo\_name) | Source repo name of the repository | `string` | n/a | yes | 141 | | [stage\_input](#input\_stage\_input) | Tags to be attached to the CodePipeline | `list(map(any))` | n/a | yes | 142 | 143 | ## Outputs 144 | 145 | | Name | Description | 146 | |------|-------------| 147 | | [codebuild\_arn](#output\_codebuild\_arn) | The ARN of the Codebuild Project | 148 | | [codebuild\_name](#output\_codebuild\_name) | The Name of the Codebuild Project | 149 | | [codepipeline\_arn](#output\_codepipeline\_arn) | The ARN of the CodePipeline | 150 | | [codepipeline\_name](#output\_codepipeline\_name) | The Name of the CodePipeline | 151 | | [iam\_arn](#output\_iam\_arn) | The ARN of the IAM Role used by the CodePipeline | 152 | | [kms\_arn](#output\_kms\_arn) | The ARN of the KMS key used in the codepipeline | 153 | | [s3\_arn](#output\_s3\_arn) | The ARN of the S3 Bucket | 154 | | [s3\_bucket\_name](#output\_s3\_bucket\_name) | The Name of the S3 Bucket | 155 | 156 | 157 | 158 | ## Security 159 | 160 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. 161 | 162 | ## License 163 | 164 | This library is licensed under the MIT-0 License. See the LICENSE file. 165 | 166 | --------------------------------------------------------------------------------