FloTorch
34 |35 | Manage and run your GenAI experiments at scale with ease 36 |
37 |
├── .codecatalyst
└── workflows
│ ├── Build_Deploy_Dev.yaml
│ └── Build_Deploy_QA.yaml
├── .github
└── ISSUE_TEMPLATE
│ ├── bug_report.yml
│ └── config.yml
├── .gitignore
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── CONTRIBUTORS.md
├── Dockerfile
├── FloTorch Colour Logo.png
├── Help_Links.MD
├── LICENSE
├── README.md
├── app
├── Dockerfile
├── __init__.py
├── common
│ ├── __init__.py
│ └── logger.py
├── configuration_validation.py
├── dependencies
│ ├── __init__.py
│ ├── database.py
│ └── s3.py
├── main.py
├── marketplace_app.Dockerfile
├── models.py
├── nginx
│ ├── auth.conf
│ ├── nginx.conf
│ ├── nginxconfig.io
│ │ ├── general.conf
│ │ ├── proxy.conf
│ │ └── security.conf
│ └── sites-enabled
│ │ └── app.conf
├── orchestrator.py
├── price_calculator.py
├── requirements.txt
├── routes
│ ├── __init__.py
│ ├── bedrock_config.py
│ ├── config.py
│ ├── cost_and_duration_calculation.py
│ ├── execution.py
│ ├── experiment.py
│ ├── expert_eval.py
│ ├── health.py
│ └── uploads.py
└── seed_data.py
├── baseclasses
└── base_classes.py
├── cfn
├── app-runner-template.yaml
├── dynamodb-template.yaml
├── ecr-repository-template.yaml
├── ecs-template.yaml
├── lambda-template.yaml
├── master-template.yaml
├── opensearch-template.yaml
├── state-machine-template.yaml
├── vpc-endpoint-template.yaml
└── vpc-template.yaml
├── config
├── __init__.py
├── config.py
└── experimental_config.py
├── constants
├── __init__.py
├── app_constants.py
├── sagemaker_constants.py
└── validation_status.py
├── core
├── __init__.py
├── chunking
│ ├── __init__.py
│ ├── fixed_chunker.py
│ └── hierarchical_chunker.py
├── dynamodb.py
├── embedding
│ ├── __init__.py
│ ├── bedrock
│ │ ├── __init__.py
│ │ ├── bedrock_embedder.py
│ │ ├── cohere_embedder.py
│ │ ├── titanv1_embedder.py
│ │ └── titanv2_embedder.py
│ ├── embedding_factory.py
│ └── sagemaker
│ │ ├── __init__.py
│ │ └── sagemaker_embedder.py
├── eval
│ ├── __init__.py
│ ├── eval_factory.py
│ └── ragas
│ │ ├── ragas_eval.py
│ │ ├── ragas_llm_eval.py
│ │ └── ragas_non_llm_eval.py
├── guardrails
│ └── bedrock_guardrails.py
├── inference
│ ├── __init__.py
│ ├── bedrock
│ │ └── bedrock_inferencer.py
│ ├── inference_factory.py
│ └── sagemaker
│ │ ├── llama_inferencer.py
│ │ └── sagemaker_inferencer.py
├── knowledgebase_vectorstore.py
├── opensearch_vectorstore.py
├── processors
│ ├── __init__.py
│ ├── chunking_processor.py
│ ├── embed_processor.py
│ ├── eval_processor.py
│ └── inference_processor.py
├── rerank
│ └── rerank.py
└── service
│ └── experimental_config_service.py
├── cover-image.png
├── dataset
├── amazon_bedrock_userguide
│ ├── Amazon_Bedrock_Dataset.pdf
│ ├── Amazon_Bedrock_Prompt.json
│ └── Amazon_bedrock_gt.json
└── medical_abstracts
│ ├── medical_abstracts.pdf
│ ├── medical_prompt.json
│ └── medical_qa_50.json
├── docs
└── pricing-calculation.md
├── evaluation
├── Dockerfile
├── eval.py
├── fargate_evaluation.Dockerfile
└── requirements.txt
├── faq.md
├── flotorch-arch.png
├── handlers
├── fargate_eval_handler.py
├── fargate_indexing_handler.py
├── fargate_retriever_handler.py
└── task_processor.py
├── indexing
├── Dockerfile
├── __init__.py
├── fargate_indexing.Dockerfile
├── indexing.py
└── requirements.txt
├── install.md
├── lambda_handlers
├── cost_handler
│ ├── Dockerfile
│ ├── cost_compute_handler.py
│ ├── pricing.py
│ ├── requirements.txt
│ └── utils.py
├── evaluation_handler.py
├── indexing_handler.py
├── opensearch_handler.py
└── retriever_handler.py
├── opensearch
├── opensearch.Dockerfile
├── opensearch_index_manager.py
└── opensearch_requirements.txt
├── provision.sh
├── release_notes
├── Release v1.0.2.md
├── Release v2.0.0.md
└── images
│ ├── Hierarchical_Chunking.png
│ ├── Indexing_Algorthm_HNSW_BQ_SQ.png
│ └── ReRanking.png
├── retriever
├── Dockerfile
├── fargate_retriever.Dockerfile
├── requirements.txt
└── retriever.py
├── test
├── FloTorch.ai TestReport.pdf
├── Integration Test Report.pdf
└── Test Plan for FloTorch.pdf
├── ui
├── .env.example
├── .gitignore
├── .ncurc.json
├── .npmrc
├── .vscode
│ └── settings.json
├── README.md
├── app
│ ├── app.config.ts
│ ├── app.vue
│ ├── assets
│ │ └── css
│ │ │ └── main.css
│ ├── components
│ │ ├── Breadcumb.vue
│ │ ├── DownloadResultsButton.vue
│ │ ├── FetchKbModels.vue
│ │ ├── FieldTooltip.vue
│ │ ├── File
│ │ │ ├── Upload.vue
│ │ │ ├── UploadKb.vue
│ │ │ └── UploadModal.vue
│ │ ├── ModelSelect.vue
│ │ ├── Page.vue
│ │ ├── Project
│ │ │ ├── Create
│ │ │ │ ├── DataStrategyStep.vue
│ │ │ │ ├── EvalStrategyStep.vue
│ │ │ │ ├── Form.vue
│ │ │ │ ├── IndexingStrategyStep.vue
│ │ │ │ └── RetrievalStrategyStep.vue
│ │ │ ├── DownloadConfigButton.vue
│ │ │ ├── Experiment
│ │ │ │ ├── Assessments.vue
│ │ │ │ ├── DetailsButton.vue
│ │ │ │ ├── DirectionalPricing.vue
│ │ │ │ ├── HumanEvaluation.vue
│ │ │ │ ├── List.vue
│ │ │ │ └── ValidList.vue
│ │ │ ├── List.vue
│ │ │ └── UploadConfigButton.vue
│ │ ├── PromptGuideHelp.vue
│ │ ├── PromptGuideSelect.vue
│ │ ├── RegionSelect.vue
│ │ └── VectorDimensionSelect.vue
│ ├── composables
│ │ ├── api.ts
│ │ ├── fileoperation.ts
│ │ ├── projects.ts
│ │ ├── shared.ts
│ │ ├── tooltip.ts
│ │ └── useShareData.ts
│ ├── layouts
│ │ └── default.vue
│ ├── pages
│ │ ├── index.vue
│ │ └── projects
│ │ │ ├── [id].vue
│ │ │ ├── [id]
│ │ │ ├── execute.vue
│ │ │ ├── experiments
│ │ │ │ ├── [experimentId].vue
│ │ │ │ └── index.vue
│ │ │ ├── humanevaluation.vue
│ │ │ ├── index.vue
│ │ │ └── validexperiments.vue
│ │ │ ├── create.vue
│ │ │ └── index.vue
│ ├── plugins
│ │ ├── 03.vuequery.ts
│ │ └── mitt.ts
│ └── utils
│ │ └── json-csv.ts
├── eslint.config.mjs
├── nuxt.config.ts
├── package.json
├── pnpm-lock.yaml
├── public
│ ├── android-chrome-192x192.png
│ ├── android-chrome-512x512.png
│ ├── apple-touch-icon.png
│ ├── favicon-16x16.png
│ ├── favicon-32x32.png
│ ├── favicon.ico
│ ├── favicon.png
│ ├── flotorch-hero.gif
│ ├── logo.png
│ ├── prompt-guide.json
│ ├── robots.txt
│ └── site.webmanifest
├── server
│ └── tsconfig.json
├── shared
│ └── types
│ │ ├── breadcumb.type.ts
│ │ ├── experiments.type.ts
│ │ ├── humaneval.type.ts
│ │ └── projects.type.ts
└── tsconfig.json
├── usage_guide.md
└── util
├── bedrock_utils.py
├── boto3_utils.py
├── date_time_utils.py
├── dynamo_utils.py
├── error_handling.py
├── guard_rails_utils.py
├── open_search_config_utils.py
├── pdf_utils.py
└── s3util.py
/.codecatalyst/workflows/Build_Deploy_Dev.yaml:
--------------------------------------------------------------------------------
1 | Name: Build_Deploy_Dev
2 | SchemaVersion: "1.0"
3 |
4 | # Only trigger when started manually.
5 | Triggers:
6 | - Type: Manual
7 |
8 | Actions:
9 | Build_a8:
10 | Identifier: aws/build@v1.0.0
11 |
12 | Inputs:
13 | Sources:
14 | - WorkflowSource
15 | Variables:
16 | # Dev environment details
17 | - Name: DEV_ENV
18 | Value: "dampen"
19 | - Name: DEV_URL
20 | Value: "https://radd4urst2.us-east-1.awsapprunner.com"
21 | - Name: DEV_USERNAME
22 | Value: "admin"
23 | - Name: DEV_PASSWORD
24 | Value: "jkhg78BKUYBKGUY7*nhiH"
25 |
26 | Outputs:
27 | AutoDiscoverReports:
28 | Enabled: true
29 | ReportNamePrefix: rpt
30 |
31 | Configuration:
32 | Steps:
33 | - Run: |
34 | echo "Deploying to Dev environment: $DEV_ENV"
35 |
36 | # 1) Log in to Amazon ECR
37 | aws ecr get-login-password --region us-east-1 \
38 | | docker login --username AWS --password-stdin 677276078734.dkr.ecr.us-east-1.amazonaws.com
39 |
40 | # 2) Build & push Docker images for Dev (paimon)
41 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-app-$DEV_ENV:latest \
42 | -f app/Dockerfile --push .
43 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-indexing-$DEV_ENV:latest \
44 | -f indexing/fargate_indexing.Dockerfile --push .
45 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-retriever-$DEV_ENV:latest \
46 | -f retriever/fargate_retriever.Dockerfile --push .
47 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-evaluation-$DEV_ENV:latest \
48 | -f evaluation/fargate_evaluation.Dockerfile --push .
49 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-runtime-$DEV_ENV:latest \
50 | -f opensearch/opensearch.Dockerfile --push .
51 | cd lambda_handlers
52 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-costcompute-$DEV_ENV:latest \
53 | -f cost_handler/Dockerfile --push .
54 | cd ..
55 |
56 | echo "Docker images for Dev ($DEV_ENV) have been pushed."
57 |
58 | Container:
59 | Registry: CODECATALYST
60 | Image: CodeCatalystLinux_x86_64:2024_03
61 |
62 | Compute:
63 | Type: EC2
64 |
65 | Environment:
66 | Name: Dev
67 |
--------------------------------------------------------------------------------
/.codecatalyst/workflows/Build_Deploy_QA.yaml:
--------------------------------------------------------------------------------
1 | Name: Build_Deploy_QA
2 | SchemaVersion: "1.0"
3 |
4 | # Only trigger when started manually.
5 | Triggers:
6 | - Type: Manual
7 |
8 | Actions:
9 | Build_a8:
10 | Identifier: aws/build@v1.0.0
11 |
12 | Inputs:
13 | Sources:
14 | - WorkflowSource
15 | Variables:
16 | # QA environment details
17 | - Name: QA_ENV
18 | Value: "iamdqa"
19 | - Name: QA_URL
20 | Value: "https://tmmqcidccg.us-east-1.awsapprunner.com"
21 | - Name: QA_USERNAME
22 | Value: "admin"
23 | - Name: QA_PASSWORD
24 | Value: "jkhrg&*ObliubUH*(8"
25 |
26 | Outputs:
27 | AutoDiscoverReports:
28 | Enabled: true
29 | ReportNamePrefix: rpt
30 |
31 | Configuration:
32 | Steps:
33 | - Run: |
34 | echo "Deploying to QA environment: $QA_ENV"
35 |
36 | # 1) Log in to Amazon ECR
37 | aws ecr get-login-password --region us-east-1 \
38 | | docker login --username AWS --password-stdin 677276078734.dkr.ecr.us-east-1.amazonaws.com
39 |
40 | # 2) Build & push Docker images for QA (paimon)
41 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-app-$QA_ENV:latest \
42 | -f app/Dockerfile --push .
43 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-indexing-$QA_ENV:latest \
44 | -f indexing/fargate_indexing.Dockerfile --push .
45 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-retriever-$QA_ENV:latest \
46 | -f retriever/fargate_retriever.Dockerfile --push .
47 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-evaluation-$QA_ENV:latest \
48 | -f evaluation/fargate_evaluation.Dockerfile --push .
49 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-runtime-$QA_ENV:latest \
50 | -f opensearch/opensearch.Dockerfile --push .
51 | cd lambda_handlers
52 | docker build -t 677276078734.dkr.ecr.us-east-1.amazonaws.com/flotorch-costcompute-$QA_ENV:latest \
53 | -f cost_handler/Dockerfile --push .
54 | cd ..
55 |
56 | echo "Docker images for QA ($QA_ENV) have been pushed."
57 |
58 | Container:
59 | Registry: CODECATALYST
60 | Image: CodeCatalystLinux_x86_64:2024_03
61 |
62 | Compute:
63 | Type: EC2
64 |
65 | Environment:
66 | Name: Dev
67 |
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/bug_report.yml:
--------------------------------------------------------------------------------
1 | name: Bug Report
2 | description: Report errors or unexpected behavior
3 | title: "[Bug]: "
4 | labels: ["bug"]
5 | assignees: []
6 | body:
7 | - type: markdown
8 | attributes:
9 | value: |
10 | Thanks for taking the time to fill out this bug report! Please ensure you've checked existing issues first.
11 |
12 | - type: input
13 | id: version
14 | attributes:
15 | label: FloTorch Version
16 | description: What version of our project are you running?
17 | placeholder: e.g., v2.0.1
18 | validations:
19 | required: true
20 |
21 | - type: textarea
22 | id: bug-description
23 | attributes:
24 | label: Bug Description
25 | description: What happened and what did you expect to happen?
26 | placeholder: Describe the bug clearly and concisely
27 | validations:
28 | required: true
29 |
30 | - type: textarea
31 | id: reproduction
32 | attributes:
33 | label: Steps to Reproduce
34 | description: How can we reproduce this issue?
35 | placeholder: |
36 | 1. Go to '...'
37 | 2. Click on '....'
38 | 3. Scroll down to '....'
39 | 4. See error
40 | validations:
41 | required: true
42 |
43 | - type: input
44 | id: browser
45 | attributes:
46 | label: Browser and Version
47 | description: Which browser are you using (if applicable)?
48 | placeholder: e.g., Chrome 120.0.6099.129
49 |
50 | - type: input
51 | id: runtime
52 | attributes:
53 | label: Runtime Version
54 | description: Which runtime version are you using?
55 | placeholder: e.g., Python 3.9.7
56 |
57 | - type: textarea
58 | id: logs
59 | attributes:
60 | label: Error Messages
61 | description: Please copy and paste any relevant error messages or stack traces
62 | render: shell
63 |
64 | - type: textarea
65 | id: additional
66 | attributes:
67 | label: Additional Context
68 | description: Add any other context about the problem here (screenshots, related issues, suggestions for fixing)
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/config.yml:
--------------------------------------------------------------------------------
1 | blank_issues_enabled: false
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | .DS_STORE
2 |
--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
1 | # Contributing to FloTorch
2 |
3 | Thank you for your interest in contributing to FloTorch. Whether you're fixing bugs, adding features, or improving documentation, your efforts are greatly appreciated. Please follow the guidelines below to ensure a smooth collaboration.
4 |
5 | # Code of Conduct
6 |
7 | By participating in this project, you agree to abide by our [Code of Conduct](CODE_OF_CONDUCT.md).
8 |
9 | # Getting Started
10 |
11 | 1. Fork the repository: Create a fork of the repository to your own GitHub account.
12 |
13 | 2. Clone your fork: Clone the forked repository to your local machine:
14 |
15 | ```git clone https://github.com/FissionAI/FloTorch.git```
16 |
17 | 3. Set up the project:
18 |
19 | - Follow the instructions in the README.md file to set up the development environment.
20 |
21 | - Install any required dependencies.
22 |
23 |
24 | ## Ways to Contribute
25 |
26 | To improve and grow the project, we need your help! Here are some ways to get involved:
27 |
28 | | Activity | Ideas |
29 | | -------- | ----- |
30 | | 👋 Discussions | Start a discussion by asking a question or making a suggestion. |
31 | | 🐛 Open an Issue | Find unhandled exceptions and bugs in the codebase. |
32 | | 📄 Documentation | Write documentation for the project. |
33 | | 🧪 Testing | Write unit tests to increase code coverage. |
34 | | 🧩 Feature Requests | Brainstorm new ideas. |
35 | | 🛠️ Code Contributions | Contribute to the codebase and submit a pull request. |
36 | | 🔢 Code Readability | Find ways to make code more readable and easier to understand. |
37 | | 🤔 Other | Anything else you can think of! |
38 |
39 | These are just a few examples, and we welcome any other ideas you may have!
40 |
41 | ## Submitting Changes
42 |
43 | 1. Fork the repository and clone it locally.
44 | 2. Create a new branch with a descriptive name. Please follow the below naming convention:
45 | - Feature branches : feature/new-feature-name
46 | - Bug fix branches : bugfix/bugfix-issue-123
47 | - Documentation branches: docs/improve-installation-guide
48 | - Refactoring branches: refactor/improve-code-structure
49 | - Test branches: test/add-unit-tests
50 | 3. Make focused changes with clear and concise commit messages.
51 | 4. Open a pull request document the changes you've made and why they're necessary.
52 | 5. Respond to code reviews from maintainers.
53 |
54 | ## Code Quality Expectations
55 |
56 | - Clear, well-documented code
57 | - Follow project style standards
58 | - Rebase onto latest master branch
59 |
60 | ## Attribution
61 |
62 | Contributors to our project will be acknowledged in the project's [CONTRIBUTORS](CONTRIBUTORS.md) file.
63 |
64 | ## License
65 |
66 | By contributing to our project, you agree to license your contributions under the project's open source license. The project's license can be found in the [LICENSE](LICENSE.txt)
67 |
68 | Thank you for your interest in contributing to readme-ai! We appreciate your help and look forward to working with you.
69 |
70 | ---
--------------------------------------------------------------------------------
/CONTRIBUTORS.md:
--------------------------------------------------------------------------------
1 | - Adil Raza [@adilraza-fission](https://github.com/adilraza-fission)
2 | - Akhila Sree
3 | - Anusha Pedireddy [LinkedIn](https://www.linkedin.com/in/anusha-peddireddy-b520346a/)
4 | - BalaSriHarsha [LinkedIn](https://www.linkedin.com/in/balasriharsha77777/)
5 | - Girish Kumar [LinkedIn](https://www.linkedin.com/in/girish-kumar-dandamudi/)
6 | - Harish Pillarisetti [LinkedIn](https://www.linkedin.com/in/harish-pillarisetti-245523100/)
7 | - Kiran George [LinkedIn](https://www.linkedin.com/in/kirangeorge96/)
8 | - Krishna Battu [@shivakrishnaah](https://github.com/shivakrishnaah)
9 | - Manisharan Ch [LinkedIn](https://www.linkedin.com/in/manisharan-ch-2335a91a3/)
10 | - Mohan Gopavaram [LinkedIn](https://www.linkedin.com/in/mohangopavaram/)
11 | - Nanda Rajasekahruni [LinkedIn](https://www.linkedin.com/in/nanda-teja-rajasekharuni-b7ab6252/)
12 | - Nikita Kumari
13 | - Prasanna Venkatesh Sridharan [LinkedIn](https://www.linkedin.com/in/prasan80/)
14 | - Prem Kasha [LinkedIn](https://www.linkedin.com/in/premkasha/)
15 | - Raj Ganesh Jayaraman [LinkedIn](https://www.linkedin.com/in/rajganeshj/)
16 | - Ravi Teja Tiruvedula [@Ravifission](https://github.com/Ravifission)
17 | - Samarendra Kandala [LinkedIn](https://www.linkedin.com/in/samarendra-kandala-a14ba490/)
18 | - Shaik Abdul Gafoor [@abdul-fission](https://github.com/abdul-fission)
19 | - Shailaja Kazipeta [LinkedIn](https://www.linkedin.com/in/shailaja-kazipeta-87447015b/)
--------------------------------------------------------------------------------
/FloTorch Colour Logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/FissionAI/FloTorch/5e27e13de07f72c23364d270bf7a086cd864469e/FloTorch Colour Logo.png
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
24 |
25 |
Please select region first
59 |23 | info 24 |
25 | 26 | -------------------------------------------------------------------------------- /ui/app/components/File/Upload.vue: -------------------------------------------------------------------------------- 1 | 94 | 95 | 96 | 97 | 98 |{{ props.description }}
30 |Indexing Cost | 23 |{{ props?.pricingInfo?.indexing_cost_estimate ? useHumanCurrencyAmount(props?.pricingInfo?.indexing_cost_estimate) : 'Unable to fetch the data'}} | 24 |
Retrieval Cost | 27 |{{ props?.pricingInfo?.retrieval_cost_estimate ? useHumanCurrencyAmount(props?.pricingInfo?.retrieval_cost_estimate) : 'Unable to fetch the data'}} | 28 |
Evaluation Cost | 31 |{{ props?.pricingInfo?.eval_cost_estimate ? useHumanCurrencyAmount(props?.pricingInfo?.eval_cost_estimate) : 'Unable to fetch the data'}} | 32 |
Your prompt guide must be a valid JSON file containing three key sections:
13 |Both the system prompt and user prompt are required
25 |26 | View sample 27 | prompt guide to understand the correct format 28 |
29 |When using N-shot prompting:
30 |35 | Manage and run your GenAI experiments at scale with ease 36 |
37 |The infrastructure provisioned for FloTorch experiments has to be explicitly deprovisioned to avoid incurring continued AWS billing.
72 |3-node cluster with r7g.2xlarge.search instance type
75 | 76 |{{useGetModelData('retrieval',data?.config?.retrieval_model)?.model_name}} : {{useGetModelData('retrieval',data?.config?.retrieval_model)?.type}}
78 |{{useGetModelData('indexing',data?.config?.embedding_model)?.model_name}} : {{useGetModelData('indexing',data?.config?.embedding_model)?.type}}
79 | 80 |