├── fts
    ├── trainer
    │   ├── __init__.py
    │   └── base.py
    ├── utils
    │   ├── __init__.py
    │   └── main.py
    ├── inference
    │   ├── __init__.py
    │   ├── base.py
    │   ├── gptq.py
    │   └── hf_model.py
    ├── processing
    │   ├── __init__.py
    │   ├── base.py
    │   └── build_dataset.py
    ├── __init__.py
    └── finetuner.py
├── docs
    ├── applications
    │   ├── enterprise.md
    │   ├── customer_support.md
    │   └── marketing_agencies.md
    ├── .DS_Store
    ├── assets
    │   ├── img
    │   │   ├── ft-logo.png
    │   │   └── tools
    │   │   │   ├── toml.png
    │   │   │   ├── output.png
    │   │   │   └── poetry_setup.png
    │   └── css
    │   │   └── extra.css
    ├── demos.md
    ├── stylesheets
    │   └── extra.css
    ├── architecture.md
    ├── metric.md
    ├── overrides
    │   └── main.html
    ├── index.md
    ├── purpose.md
    ├── hiring.md
    ├── faq.md
    ├── ft
    │   ├── gptq_inference.md
    │   ├── index.md
    │   ├── inference.md
    │   └── finetuner.md
    ├── design.md
    ├── contributing.md
    ├── bounties.md
    ├── roadmap.md
    └── flywheel.md
├── .DS_Store
├── images
    ├── ft-logo.png
    └── agorabanner.png
├── inference.py
├── .pre-commit-config.yaml
├── requirements.txt
├── .readthedocs.yml
├── example.py
├── .github
    ├── workflows
    │   ├── pull-request-links.yml
    │   ├── docs.yml
    │   ├── welcome.yml
    │   ├── label.yml
    │   ├── pylint.yml
    │   ├── python-publish.yml
    │   ├── stale.yml
    │   ├── unit-test.yml
    │   ├── publish.yml
    │   └── test.yml
    ├── dependabot.yml
    ├── ISSUE_TEMPLATE
    │   ├── feature_request.md
    │   └── bug_report.md
    ├── FUNDING.yml
    └── PULL_REQUEST_TEMPLATE.yml
├── pyproject.toml
├── Makefile
├── playground
    └── llama2_english.py
├── mkdocs.yml
├── .gitignore
├── README.md
└── LICENSE


/fts/trainer/__init__.py:
--------------------------------------------------------------------------------
1 | 


--------------------------------------------------------------------------------
/fts/utils/__init__.py:
--------------------------------------------------------------------------------
1 | 


--------------------------------------------------------------------------------
/fts/inference/__init__.py:
--------------------------------------------------------------------------------
1 | 


--------------------------------------------------------------------------------
/fts/processing/__init__.py:
--------------------------------------------------------------------------------
1 | 


--------------------------------------------------------------------------------
/docs/applications/enterprise.md:
--------------------------------------------------------------------------------
1 | 


--------------------------------------------------------------------------------
/.DS_Store:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/.DS_Store


--------------------------------------------------------------------------------
/docs/.DS_Store:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/docs/.DS_Store


--------------------------------------------------------------------------------
/images/ft-logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/images/ft-logo.png


--------------------------------------------------------------------------------
/images/agorabanner.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/images/agorabanner.png


--------------------------------------------------------------------------------
/docs/assets/img/ft-logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/docs/assets/img/ft-logo.png


--------------------------------------------------------------------------------
/docs/assets/img/tools/toml.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/docs/assets/img/tools/toml.png


--------------------------------------------------------------------------------
/docs/assets/img/tools/output.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/docs/assets/img/tools/output.png


--------------------------------------------------------------------------------
/docs/demos.md:
--------------------------------------------------------------------------------
1 | # Demo Ideas
2 | 
3 | * GPT-4
4 | * Andromeda
5 | * Kosmos
6 | * LongNet
7 | * Text to video diffusion
8 | * Nebula
9 | 


--------------------------------------------------------------------------------
/docs/assets/img/tools/poetry_setup.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kyegomez/Finetuning-Suite/HEAD/docs/assets/img/tools/poetry_setup.png


--------------------------------------------------------------------------------
/docs/stylesheets/extra.css:
--------------------------------------------------------------------------------
1 | :root {
2 |     --md-primary-fg-color:        #8315F9;
3 |     --md-accent-fg-color:         #00FFCE;
4 |   }


--------------------------------------------------------------------------------
/docs/assets/css/extra.css:
--------------------------------------------------------------------------------
1 | .md-typeset__table {
2 |    min-width: 100%;
3 | }
4 | 
5 | .md-typeset table:not([class]) {
6 |     display: table;
7 | }


--------------------------------------------------------------------------------
/inference.py:
--------------------------------------------------------------------------------
1 | from fts import Inference
2 | 
3 | model = Inference(
4 |     model_id="georgesung/llama2_7b_chat_uncensored",
5 |     quantized=True
6 | )
7 | 
8 | model.run("What is your name")


--------------------------------------------------------------------------------
/.pre-commit-config.yaml:
--------------------------------------------------------------------------------
1 | - repo: https://github.com/astral-sh/ruff-pre-commit
2 |   # Ruff version.
3 |   rev: v0.0.286
4 |   hooks:
5 |     - id: ruff
6 |       args: [--fix, --exit-non-zero-on-fix]


--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
 1 | transformers
 2 | bitsandbytes
 3 | accelerate
 4 | datasets
 5 | rich
 6 | tensorboard
 7 | wandb
 8 | tokenizers
 9 | optimum
10 | 
11 | 
12 | mkdocs
13 | mkdocs-material
14 | mkdocs-glightbox
15 | 


--------------------------------------------------------------------------------
/docs/architecture.md:
--------------------------------------------------------------------------------
1 | # Architecture 
2 | * Simple file structure
3 | * Fluid API 
4 | * Useful error handling that provides potential solutions and root cause error understanding
5 | * nn, tokenizers, models, training
6 | * 


--------------------------------------------------------------------------------
/.readthedocs.yml:
--------------------------------------------------------------------------------
 1 | version: 2
 2 | 
 3 | build:
 4 |   os: ubuntu-22.04
 5 |   tools:
 6 |     python: "3.11"
 7 | 
 8 | mkdocs:
 9 |   configuration: mkdocs.yml
10 | 
11 | python:
12 |    install:
13 |    - requirements: requirements.txt


--------------------------------------------------------------------------------
/docs/metric.md:
--------------------------------------------------------------------------------
1 | # The Golden Metric: 
2 | 
3 | * We need to figure out a single metric that determines if we're accomplishing our goal with zeta which is to build zetascale superintelligent AI models as fast as possible with minimal code.
4 | 
5 | 


--------------------------------------------------------------------------------
/fts/__init__.py:
--------------------------------------------------------------------------------
1 | from fts.finetuner import FineTuner
2 | from fts.inference.hf_model import Inference
3 | 
4 | from fts.processing.base import Preprocessor, DefaultPreprocessor
5 | from fts.trainer.base import TrainerConfiguration, DefaultTrainerConfig
6 | 
7 | from fts.processing.build_dataset import BuildDataset
8 | 


--------------------------------------------------------------------------------
/docs/overrides/main.html:
--------------------------------------------------------------------------------
1 | {% extends "base.html" %}
2 | 
3 | <!--https://squidfunk.github.io/mkdocs-material/customization/#overriding-blocks-->
4 | 
5 | {% block announce %}
6 |   <div style="text-align:center">
7 |     <a href="https://github.com/kyegomez/zeta">Star and contribute</a> to Zeta on GitHub!
8 |   </div>
9 | {% endblock %}


--------------------------------------------------------------------------------
/example.py:
--------------------------------------------------------------------------------
 1 | from fts import FineTuner
 2 | 
 3 | model_id="google/flan-t5-xxl"
 4 | 
 5 | dataset_name="samsum"
 6 | 
 7 | finetune = FineTuner(
 8 |     model_id=model_id,
 9 |     dataset_name="samsum",
10 |     max_length=150,
11 |     lora_r=16,
12 |     lora_alpha=32,
13 |     quantize=True
14 | )
15 | 
16 | 
17 | finetune.train


--------------------------------------------------------------------------------
/.github/workflows/pull-request-links.yml:
--------------------------------------------------------------------------------
 1 | name: readthedocs/actions
 2 | on:
 3 |   pull_request_target:
 4 |     types:
 5 |       - opened
 6 |     paths:
 7 |       - "docs/**"
 8 | 
 9 | permissions:
10 |   pull-requests: write
11 | 
12 | jobs:
13 |   pull-request-links:
14 |     runs-on: ubuntu-latest
15 |     steps:
16 |       - uses: readthedocs/actions/preview@v1
17 |         with:
18 |           project-slug: ft


--------------------------------------------------------------------------------
/.github/dependabot.yml:
--------------------------------------------------------------------------------
 1 | # https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically/configuration-options-for-dependency-updates
 2 | 
 3 | version: 2
 4 | updates:
 5 |   - package-ecosystem: "github-actions"
 6 |     directory: "/"
 7 |     schedule:
 8 |       interval: "weekly"
 9 | 
10 |   - package-ecosystem: "pip"
11 |     directory: "/"
12 |     schedule:
13 |       interval: "weekly"
14 | 
15 | 


--------------------------------------------------------------------------------
/fts/utils/main.py:
--------------------------------------------------------------------------------
 1 | 
 2 | 
 3 | def print_trainable_parameters(model):
 4 |     trainable_params = 0
 5 |     all_param = 0
 6 |     for _, param in model.named_parameters():
 7 |         all_param += param.numel()
 8 |         if param.requires_grad:
 9 |             trainable_params += param.numel()
10 |         
11 |     print(
12 |         f"Trainable params: {trainable_params} || all params {all_param} || trainable: {100 * trainable_params / all_param}"
13 |     )


--------------------------------------------------------------------------------
/.github/workflows/docs.yml:
--------------------------------------------------------------------------------
 1 | name: Docs WorkFlow
 2 | 
 3 | on:
 4 |   push:
 5 |     branches:
 6 |       - master
 7 |       - main
 8 |       - develop
 9 | jobs:
10 |   deploy:
11 |     runs-on: ubuntu-latest
12 |     steps:
13 |       - uses: actions/checkout@v3
14 |       - uses: actions/setup-python@v5
15 |         with:
16 |           python-version: 3.x
17 |       - run: pip install mkdocs-material
18 |       - run: pip install "mkdocstrings[python]"
19 |       - run: mkdocs gh-deploy --force


--------------------------------------------------------------------------------
/docs/index.md:
--------------------------------------------------------------------------------
 1 | # Finetuning Suite Docs
 2 | 
 3 | Welcome to Finetuning Suite's Documentation!
 4 | 
 5 | Finetuning Suite  is a modular framework that enables for seamless, reliable, and fluid finetuning and inference
 6 | 
 7 | ## Finetuning Suite
 8 | 
 9 | <!-- ![Zeta Banner](docs/assets/img/zetascale.png) -->
10 | 
11 | Finetuning Suite  is a modular framework that enables for seamless, reliable, and fluid finetuning and inference
12 | [Click here for Finetuning Suite Documentation →](ft/)
13 | 
14 | 
15 | 


--------------------------------------------------------------------------------
/.github/workflows/welcome.yml:
--------------------------------------------------------------------------------
 1 | name: Welcome WorkFlow
 2 | 
 3 | on:
 4 |   issues:
 5 |     types: [opened]
 6 |   pull_request_target:
 7 |     types: [opened]
 8 | 
 9 | jobs:
10 |   build:
11 |     name: 👋 Welcome
12 |     runs-on: ubuntu-latest
13 |     steps:
14 |       - uses: actions/first-interaction@v1.3.0
15 |         with:
16 |           repo-token: ${{ secrets.GITHUB_TOKEN }}
17 |           issue-message: "Hello there, thank you for opening an Issue ! 🙏🏻 The team was notified and they will get back to you asap."
18 |           pr-message:  "Hello there, thank you for opening an PR ! 🙏🏻 The team was notified and they will get back to you asap."


--------------------------------------------------------------------------------
/.github/workflows/label.yml:
--------------------------------------------------------------------------------
 1 | # This workflow will triage pull requests and apply a label based on the
 2 | # paths that are modified in the pull request.
 3 | #
 4 | # To use this workflow, you will need to set up a .github/labeler.yml
 5 | # file with configuration.  For more information, see:
 6 | # https://github.com/actions/labeler
 7 | 
 8 | name: Labeler
 9 | on: [pull_request_target]
10 | 
11 | jobs:
12 |   label:
13 | 
14 |     runs-on: ubuntu-latest
15 |     permissions:
16 |       contents: read
17 |       pull-requests: write
18 | 
19 |     steps:
20 |     - uses: actions/labeler@v5
21 |       with:
22 |         repo-token: "${{ secrets.GITHUB_TOKEN }}"
23 | 


--------------------------------------------------------------------------------
/.github/workflows/pylint.yml:
--------------------------------------------------------------------------------
 1 | name: Pylint
 2 | 
 3 | on: [push]
 4 | 
 5 | jobs:
 6 |   build:
 7 |     runs-on: ubuntu-latest
 8 |     strategy:
 9 |       matrix:
10 |         python-version: ["3.8", "3.9", "3.10"]
11 |     steps:
12 |     - uses: actions/checkout@v3
13 |     - name: Set up Python ${{ matrix.python-version }}
14 |       uses: actions/setup-python@v5
15 |       with:
16 |         python-version: ${{ matrix.python-version }}
17 |     - name: Install dependencies
18 |       run: |
19 |         python -m pip install --upgrade pip
20 |         pip install pylint
21 |     - name: Analysing the code with pylint
22 |       run: |
23 |         pylint $(git ls-files '*.py')
24 | 


--------------------------------------------------------------------------------
/pyproject.toml:
--------------------------------------------------------------------------------
 1 | [tool.poetry]
 2 | name = "ft-suite"
 3 | version = "0.1.7"
 4 | description = "A fine-tuning suite based on Transformers and LoRA."
 5 | authors = ["Kye Gomez <kye@apac.ai>"]
 6 | license = "MIT"  
 7 | packages = [
 8 |     { include = "fts" },
 9 |     { include = "fts/**/*.py" },
10 | ]
11 | 
12 | [tool.poetry.dependencies]
13 | python = "^3.7"
14 | torch = "*" 
15 | transformers = "*" 
16 | datasets = "*" 
17 | peft = "*"  
18 | accelerate = "*"
19 | optimum = "*"
20 | bitsandbytes = "*"
21 | 
22 | 
23 | [tool.poetry.dev-dependencies]
24 | pytest = "^5.2"
25 | 
26 | [build-system]
27 | requires = ["poetry-core>=1.0.0"]
28 | build-backend = "poetry.core.masonry.api"
29 | 


--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/feature_request.md:
--------------------------------------------------------------------------------
 1 | ---
 2 | name: Feature request
 3 | about: Suggest an idea for this project
 4 | title: ''
 5 | labels: ''
 6 | assignees: 'kyegomez'
 7 | 
 8 | ---
 9 | 
10 | **Is your feature request related to a problem? Please describe.**
11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
12 | 
13 | **Describe the solution you'd like**
14 | A clear and concise description of what you want to happen.
15 | 
16 | **Describe alternatives you've considered**
17 | A clear and concise description of any alternative solutions or features you've considered.
18 | 
19 | **Additional context**
20 | Add any other context or screenshots about the feature request here.
21 | 


--------------------------------------------------------------------------------
/.github/FUNDING.yml:
--------------------------------------------------------------------------------
 1 | # These are supported funding model platforms
 2 | 
 3 | github: [kyegomez]
 4 | patreon: # Replace with a single Patreon username
 5 | open_collective: # Replace with a single Open Collective username
 6 | ko_fi: # Replace with a single Ko-fi username
 7 | tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
 8 | community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
 9 | liberapay: # Replace with a single Liberapay username
10 | issuehunt: # Replace with a single IssueHunt username
11 | otechie: # Replace with a single Otechie username
12 | lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
13 | custom: #Nothing
14 | 


--------------------------------------------------------------------------------
/.github/workflows/python-publish.yml:
--------------------------------------------------------------------------------
 1 | 
 2 | name: Upload Python Package
 3 | 
 4 | on:
 5 |   release:
 6 |     types: [published]
 7 | 
 8 | permissions:
 9 |   contents: read
10 | 
11 | jobs:
12 |   deploy:
13 | 
14 |     runs-on: ubuntu-latest
15 | 
16 |     steps:
17 |     - uses: actions/checkout@v3
18 |     - name: Set up Python
19 |       uses: actions/setup-python@v5
20 |       with:
21 |         python-version: '3.x'
22 |     - name: Install dependencies
23 |       run: |
24 |         python -m pip install --upgrade pip
25 |         pip install build
26 |     - name: Build package
27 |       run: python -m build
28 |     - name: Publish package
29 |       uses: pypa/gh-action-pypi-publish@2f6f737ca5f74c637829c0f5c3acd0e29ea5e8bf
30 |       with:
31 |         user: __token__
32 |         password: ${{ secrets.PYPI_API_TOKEN }}


--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/bug_report.md:
--------------------------------------------------------------------------------
 1 | ---
 2 | name: Bug report
 3 | about: Create a detailed report on the bug and it's root cause. Conduct root cause error analysis
 4 | title: "[BUG] "
 5 | labels: bug
 6 | assignees: kyegomez
 7 | 
 8 | ---
 9 | 
10 | **Describe the bug**
11 | A clear and concise description of what the bug is and what the main root cause error is. Test very thoroughly before submitting.
12 | 
13 | **To Reproduce**
14 | Steps to reproduce the behavior:
15 | 1. Go to '...'
16 | 2. Click on '....'
17 | 3. Scroll down to '....'
18 | 4. See error
19 | 
20 | **Expected behavior**
21 | A clear and concise description of what you expected to happen.
22 | 
23 | **Screenshots**
24 | If applicable, add screenshots to help explain your problem.
25 | 
26 | **Additional context**
27 | Add any other context about the problem here.
28 | 


--------------------------------------------------------------------------------
/fts/inference/base.py:
--------------------------------------------------------------------------------
 1 | from abc import ABC, abstractmethod
 2 | 
 3 | class InferenceHandler(ABC):
 4 |     @abstractmethod
 5 |     def run(
 6 |         self, 
 7 |         prompt_text=None, 
 8 |         model=None, 
 9 |         tokenizer=None, 
10 |         device=None, 
11 |         max_length = None
12 |     ):
13 |         pass
14 | 
15 | 
16 | class DefaultInferenceHandler(InferenceHandler):
17 |     def run(
18 |             self, 
19 |             prompt_text, 
20 |             model, 
21 |             tokenizer, 
22 |             device, 
23 |             max_length
24 |         ):
25 |         inputs = tokenizer.encode(prompt_text, return_tensors="pt").to(self.device)
26 |         outputs = model.run(inputs, max_length=max_length, do_sample=True)
27 |         return tokenizer.decode(outputs[0], skip_special_tokens=True)
28 |     
29 | 
30 | 
31 | 


--------------------------------------------------------------------------------
/.github/workflows/stale.yml:
--------------------------------------------------------------------------------
 1 | # This workflow warns and then closes issues and PRs that have had no activity for a specified amount of time.
 2 | #
 3 | # You can adjust the behavior by modifying this file.
 4 | # For more information, see:
 5 | # https://github.com/actions/stale
 6 | name: Mark stale issues and pull requests
 7 | 
 8 | on:
 9 |   schedule:
10 |   - cron: '26 12 * * *'
11 | 
12 | jobs:
13 |   stale:
14 | 
15 |     runs-on: ubuntu-latest
16 |     permissions:
17 |       issues: write
18 |       pull-requests: write
19 | 
20 |     steps:
21 |     - uses: actions/stale@v9
22 |       with:
23 |         repo-token: ${{ secrets.GITHUB_TOKEN }}
24 |         stale-issue-message: 'Stale issue message'
25 |         stale-pr-message: 'Stale pull request message'
26 |         stale-issue-label: 'no-issue-activity'
27 |         stale-pr-label: 'no-pr-activity'


--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
 1 | .PHONY: style check_code_quality
 2 | 
 3 | export PYTHONPATH = .
 4 | check_dirs := src
 5 | 
 6 | style:
 7 | 	black  $(check_dirs)
 8 | 	isort --profile black $(check_dirs)
 9 | 
10 | check_code_quality:
11 | 	black --check $(check_dirs)
12 | 	isort --check-only --profile black $(check_dirs)
13 | 	# stop the build if there are Python syntax errors or undefined names
14 | 	flake8 $(check_dirs) --count --select=E9,F63,F7,F82 --show-source --statistics
15 | 	# exit-zero treats all errors as warnings. E203 for black, E501 for docstring, W503 for line breaks before logical operators 
16 | 	flake8 $(check_dirs) --count --max-line-length=88 --exit-zero  --ignore=D --extend-ignore=E203,E501,W503  --statistics
17 | 	
18 | publish:
19 | 	python setup.py sdist bdist_wheel
20 | 	twine upload -r testpypi dist/* -u ${PYPI_USERNAME} -p ${PYPI_TEST_PASSWORD} --verbose 
21 | 	twine check dist/*
22 | 	twine upload dist/* -u ${PYPI_USERNAME} -p ${PYPI_PASSWORD} --verbose 


--------------------------------------------------------------------------------
/.github/workflows/unit-test.yml:
--------------------------------------------------------------------------------
 1 | name: build
 2 | 
 3 | on:
 4 |   push:
 5 |     branches: [ main ]
 6 |   pull_request:
 7 |     branches: [ main ]
 8 | 
 9 | jobs:
10 | 
11 |   build:
12 | 
13 |     runs-on: ubuntu-latest
14 | 
15 |     steps:
16 |     - uses: actions/checkout@v3
17 | 
18 |     - name: Setup Python
19 |       uses: actions/setup-python@v5
20 |       with:
21 |         python-version: '3.10'
22 | 
23 |     - name: Install dependencies
24 |       run: pip install -r requirements.txt
25 | 
26 |     - name: Run Python unit tests
27 |       run: python3 -m unittest tests/finetuning_suite
28 | 
29 |     - name: Verify that the Docker image for the action builds
30 |       run: docker build . --file Dockerfile
31 | 
32 |     - name: Integration test 1
33 |       uses: ./
34 |       with:
35 |         input-one: something
36 |         input-two: true
37 | 
38 |     - name: Integration test 2
39 |       uses: ./
40 |       with:
41 |         input-one: something else
42 |         input-two: false
43 | 
44 |     - name: Verify integration test results
45 |       run: python3 -m unittest unittesting/finetuning_suite
46 | 


--------------------------------------------------------------------------------
/.github/PULL_REQUEST_TEMPLATE.yml:
--------------------------------------------------------------------------------
 1 | <!-- Thank you for contributing to Finetuning Zeta!
 2 | 
 3 | Replace this comment with:
 4 |   - Description: a description of the change, 
 5 |   - Issue: the issue # it fixes (if applicable),
 6 |   - Dependencies: any dependencies required for this change,
 7 |   - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
 8 |   - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!
 9 | 
10 | If you're adding a new integration, please include:
11 |   1. a test for the integration, preferably unit tests that do not rely on network access,
12 |   2. an example notebook showing its use.
13 | 
14 | Maintainer responsibilities:
15 |   - nn / Misc / if you don't know who to tag: kye@apac.ai
16 |   - tokenizers: kye@apac.ai
17 |   - training / Prompts: kye@apac.ai
18 |   - models: kye@apac.ai
19 | 
20 | If no one reviews your PR within a few days, feel free to kye@apac.ai
21 | 
22 | See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/kyegomez/Finetuning-Suite


--------------------------------------------------------------------------------
/fts/processing/base.py:
--------------------------------------------------------------------------------
 1 | from abc import ABC, abstractmethod
 2 | 
 3 | class Preprocessor(ABC):
 4 |     def __init__(self, tokenizer):
 5 |         self.tokenizer = tokenizer
 6 | 
 7 |     @abstractmethod
 8 |     def preprocess_function(self, sample, padding="max_length"):
 9 |         pass
10 | 
11 | 
12 | # Step 2: Default Preprocessor
13 | class DefaultPreprocessor(Preprocessor):
14 | 
15 |     def preprocess_function(
16 |             self, 
17 |             sample, 
18 |             padding="max_length", 
19 |             max_source_length=None, 
20 |             max_target_length=None
21 |         ):
22 |         inputs = ["prompt" + item for item in sample["act"]]
23 |         model_inputs = self.tokenizer(inputs, max_length=max_source_length, padding=padding, truncation=True)
24 |         labels = self.tokenizer(text_target=sample["prompt"], max_length=max_target_length, padding=padding, truncation=True)
25 |         if padding == "max_length":
26 |             labels["input_ids"] = [
27 |                 [(l if l != self.tokenizer.pad_token_id else -100) for l in label] for label in labels["input_ids"]
28 |             ]
29 |         model_inputs["labels"] = labels["input_ids"]
30 |         return model_inputs


--------------------------------------------------------------------------------
/.github/workflows/publish.yml:
--------------------------------------------------------------------------------
 1 | name: Supervision Releases to PyPi
 2 | on:
 3 |   push:
 4 |     tags:
 5 |       - '[0-9]+.[0-9]+[0-9]+.[0-9]'
 6 |       - '[0-9]+.[0-9]+[0-9]+.[0-9]'
 7 |       - '[0-9]+.[0-9]+[0-9]+.[0-9]'
 8 | 
 9 |   # Allows you to run this workflow manually from the Actions tab
10 |   workflow_dispatch:
11 | 
12 | jobs:
13 |   build:
14 |     runs-on: ubuntu-latest
15 |     strategy:
16 |       matrix:
17 |         python-version: [3.8]
18 |     steps:
19 |       - name: 🛎️ Checkout
20 |         uses: actions/checkout@v3
21 |         with:
22 |           ref: ${{ github.head_ref }}
23 |       - name: 🐍 Set up Python ${{ matrix.python-version }}
24 |         uses: actions/setup-python@v5
25 |         with:
26 |           python-version: ${{ matrix.python-version }}
27 | 
28 |       - name:  🏗️ Build source and wheel distributions
29 |         run: |
30 |           python -m pip install --upgrade build twine
31 |           python -m build
32 |           twine check --strict dist/*
33 |       - name: 🚀 Publish to PyPi
34 |         uses: pypa/gh-action-pypi-publish@release/v1
35 |         with:
36 |           user: ${{ secrets.PYPI_USERNAME }}
37 |           password: ${{ secrets.PYPI_PASSWORD }}
38 |       - name: 🚀 Publish to Test-PyPi
39 |         uses: pypa/gh-action-pypi-publish@release/v1
40 |         with:
41 |           repository-url: https://test.pypi.org/legacy/
42 |           user: ${{ secrets.PYPI_TEST_USERNAME }}
43 |           password: ${{ secrets.PYPI_TEST_PASSWORD }}


--------------------------------------------------------------------------------
/playground/llama2_english.py:
--------------------------------------------------------------------------------
 1 | from datasets import load_dataset
 2 | from transformers import AutoTokenizer
 3 | 
 4 | from fts.finetuner import FineTuner
 5 | 
 6 | tokenizer = AutoTokenizer.from_pretrained("Phind/Phind-CodeLlama-34B-v1")
 7 | 
 8 | def data_preprocessing(dataset="Abirate/english_quotes"):
 9 |     data = load_dataset(dataset)
10 |     data = data.map(
11 |         lambda samples: tokenizer(samples["quote"]), batched=True
12 |     )
13 | 
14 | 
15 | def trainer(model):
16 |     import transformers
17 | 
18 |     # needed for gpt-neo-x tokenizer
19 |     tokenizer.pad_token = tokenizer.eos_token
20 | 
21 |     trainer = transformers.Trainer(
22 |         model=model,
23 |         train_dataset=data_preprocessing["train"],
24 |         args=transformers.TrainingArguments(
25 |             per_device_train_batch_size=1,
26 |             gradient_accumulation_steps=4,
27 |             warmup_steps=2,
28 |             max_steps=10,
29 |             learning_rate=2e-4,
30 |             fp16=True,
31 |             logging_steps=1,
32 |             output_dir="outputs",
33 |             optim="paged_adamw_8bit"
34 |         ),
35 |         data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),
36 |     )
37 |     model.config.use_cache = False  # silence the warnings. Please re-enable for inference!
38 |     trainer.train()
39 | 
40 | 
41 | FineTuner(
42 |     model_id="Phind/Phind-CodeLlama-34B-v1",
43 |     preprocessor=data_preprocessing,
44 |     trainer_config=trainer
45 | )
46 | 
47 | 


--------------------------------------------------------------------------------
/.github/workflows/test.yml:
--------------------------------------------------------------------------------
 1 | name: test
 2 | 
 3 | on:
 4 |   push:
 5 |     branches: [master]
 6 |   pull_request:
 7 |   workflow_dispatch:
 8 | 
 9 | env:
10 |   POETRY_VERSION: "1.4.2"
11 | 
12 | jobs:
13 |   build:
14 |     runs-on: ubuntu-latest
15 |     strategy:
16 |       matrix:
17 |         python-version:
18 |           - "3.8"
19 |           - "3.9"
20 |           - "3.10"
21 |           - "3.11"
22 |         test_type:
23 |           - "core"
24 |           - "extended"
25 |     name: Python ${{ matrix.python-version }} ${{ matrix.test_type }}
26 |     steps:
27 |       - uses: actions/checkout@v3
28 |       - name: Set up Python ${{ matrix.python-version }}
29 |         uses: "./.github/actions/poetry_setup"
30 |         with:
31 |           python-version: ${{ matrix.python-version }}
32 |           poetry-version: "1.4.2"
33 |           cache-key: ${{ matrix.test_type }}
34 |           install-command: |
35 |               if [ "${{ matrix.test_type }}" == "core" ]; then
36 |                 echo "Running core tests, installing dependencies with poetry..."
37 |                 poetry install
38 |               else
39 |                 echo "Running extended tests, installing dependencies with poetry..."
40 |                 poetry install -E extended_testing
41 |               fi
42 |       - name: Run ${{matrix.test_type}} tests
43 |         run: |
44 |           if [ "${{ matrix.test_type }}" == "core" ]; then
45 |             make test
46 |           else
47 |             make extended_tests
48 |           fi
49 |         shell: bash


--------------------------------------------------------------------------------
/fts/trainer/base.py:
--------------------------------------------------------------------------------
 1 | from abc import ABC, abstractmethod
 2 | from peft import LoraConfig, TaskType, get_peft_model
 3 | from transformers import (
 4 |     DataCollatorForSeq2Seq,
 5 |     Seq2SeqTrainingArguments,
 6 | )
 7 | 
 8 | 
 9 | class TrainerConfiguration(ABC):
10 |     @abstractmethod
11 |     def configure(self, model, tokenizer, output_dir, num_train_epochs, *args, **kwargs):
12 |         """
13 |         Configures the model collator, and training arguments
14 | 
15 |         Returns:
16 |             tuple: (configured model, data_collator, training_args)
17 |         """
18 |         pass
19 | 
20 | 
21 | class DefaultTrainerConfig(TrainerConfiguration):
22 | 
23 |     def configure(self, model, tokenizer, output_dir, num_train_epochs, *args, **kwargs):
24 |         lora_config = LoraConfig(
25 |             r=16,
26 |             lora_alpha=32,
27 |             target_modules=["q", "v"],
28 |             bias="none",
29 |             task_type=TaskType.SEQ_2_SEQ_LM,
30 |         )
31 |         model = get_peft_model(model, lora_config)
32 |         
33 |         data_collator = DataCollatorForSeq2Seq(tokenizer, model=model,  label_pad_token_id=-100, pad_to_multiple_of=8 )
34 | 
35 |         training_args = Seq2SeqTrainingArguments(
36 |             output_dir=output_dir,
37 |             auto_find_batch_size=True,
38 |             learning_rate=1e-3,
39 |             num_train_epochs=num_train_epochs,
40 |             logging_dir=f"{output_dir}/logs",
41 |             logging_strategy="steps",
42 |             logging_steps=500,
43 |             save_strategy="no",
44 |             report_to="tensorboard"
45 |         )
46 | 
47 |         return model, data_collator, training_args
48 |     


--------------------------------------------------------------------------------
/docs/purpose.md:
--------------------------------------------------------------------------------
 1 | # Zeta's Purpose
 2 | 
 3 | 
 4 | Eevery once in a while, a revolutionary project comes along that changes everything.
 5 | 
 6 | A landscape cluttered by rigid frameworks, plagued by inefficiencies, and where developers - our brightest minds - are bogged down by limitations.
 7 | 
 8 | Now, imagine a world where harnessing the power of state-of-the-art models isn't just possible... it's simple. A world where efficiency doesn’t sacrifice safety, and where your ideas are bounded only by your imagination. We should be living in this world. But we aren't.
 9 | 
10 | 
11 | Because Zeta is what's missing.
12 | 
13 | 
14 | The challenge? Creating a framework that's not just another tool, but a revolution.
15 | 
16 | To bridge this gap, one would need to optimize at the foundational level, prioritize user experience, and introduce a design philosophy that future-proofs. It's colossal. And until now, no one's even come close.
17 | 
18 | 
19 | But there’s an enormous opportunity here. An opportunity that promises not just recognition but the power to redefine an industry. And, the key to unlocking this future? It's been with us all along.
20 | 
21 | 
22 | Insight.
23 | 
24 | 
25 | Introducing... Zeta.
26 | 
27 | 
28 | Our secret? Fluidity.
29 | 
30 | It’s a philosophy that values modularity, reliability, usability, and unmatched speed. 
31 | 
32 | But more than that, it's a commitment to evolution, to pushing boundaries, to never settling.
33 | 
34 | 
35 | Why are we the best to execute this vision? 
36 | 
37 | Because we've been there from the start. 
38 | 
39 | We've seen the challenges, felt the frustrations, and now, we're poised to lead the revolution. 
40 | 
41 | We’ve done it before, and with Zeta, we’re doing it again.
42 | 
43 | 
44 | Zeta isn’t just the next step. It's a leap into the future.
45 | 
46 | Zeta is the future of AI.
47 | 
48 | 


--------------------------------------------------------------------------------
/fts/inference/gptq.py:
--------------------------------------------------------------------------------
 1 | import torch
 2 | from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
 3 | 
 4 | class GPTQInference:
 5 |     def __init__(
 6 |         self,
 7 |         model_id,
 8 |         quantization_config_bits: int = 4,
 9 |         quantization_config_dataset: str = None,
10 |         max_length: int = 500
11 |     ):
12 |         self.model_id = model_id
13 |         self.quantization_config_bits = quantization_config_bits
14 |         self.quantization_config_dataset = quantization_config_dataset
15 |         self.max_length = max_length
16 | 
17 | 
18 |         self.tokenizer = AutoTokenizer.from_pretrained(self.model_id)
19 |         self.quantization_config = GPTQConfig(
20 |             bits=self.quantization_config_bits,
21 |             dataset=quantization_config_dataset,
22 |             tokenizer=self.tokenizer
23 |         )
24 | 
25 |         self.model = AutoModelForCausalLM.from_pretrained(
26 |             self.model_id,
27 |             device_map="auto",
28 |             quantization_config=self.quantization_config
29 |         )
30 | 
31 |     def run(
32 |             self, 
33 |             prompt: str,
34 |             # max_length: int =x None
35 |         ):
36 |         # max_length = max_length if max_length else self.max_length
37 | 
38 |         try:
39 |             inputs = self.tokenizer.encode(
40 |                 prompt,
41 |                 return_tensors="pt"
42 |             ).to(self.device)
43 | 
44 |             with torch.no_grad():
45 |                 outputs = self.model.generate(
46 |                     inputs,
47 |                     max_length=self.max_length,
48 |                     do_sample=True
49 |                 )
50 | 
51 |             return self.tokenizer.decode(
52 |                 outputs[0],
53 |                 skip_special_tokens=True
54 |             )
55 |         
56 |         except Exception as error:
57 |             print(f"Error: {error} in inference mode, please change the inference logic or try again")
58 |             raise


--------------------------------------------------------------------------------
/fts/processing/build_dataset.py:
--------------------------------------------------------------------------------
 1 | import argparse
 2 | import multiprocessing
 3 | from itertools import chain
 4 | 
 5 | from datasets import load_dataset
 6 | 
 7 | from kosmosx.model import KosmosTokenizer
 8 | 
 9 | 
10 | class BuildDataset:
11 |     def __init__(self, seed=42, seq_len=8192, hf_account="YOUR HUGGINGFACE API KEY", dataset_name="uggingFaceM4/VQAv2"):
12 |         self.SEED = seed
13 |         self.SEQ_LEN = seq_len
14 |         self.NUM_CPU = multiprocessing.cpu_count()
15 |         self.HF_ACCOUNT_REPO = hf_account
16 |         self.DATASET_NAME = dataset_name
17 |         self.tokenizer = KosmosTokenizer.tokenize
18 | 
19 |     def tokenize_function(self, example):
20 |         return self.tokenizer([t + self.tokenizer.eos_token for t in example["text"]])
21 | 
22 |     def group_texts(self, examples):
23 |         concatenated_examples = {k: list(chain(*examples[k])) for k in examples.keys()}
24 |         total_length = len(concatenated_examples[list(examples.keys())[0]])
25 |         if total_length >= self.SEQ_LEN:
26 |             total_length = (total_length // self.SEQ_LEN) * self.SEQ_LEN
27 |         result = {
28 |             k: [t[i : i + self.SEQ_LEN] for i in range(0, total_length, self.SEQ_LEN)]
29 |             for k, t in concatenated_examples.items()
30 |         }
31 |         return result
32 | 
33 |     def build(self):
34 |         train_dataset = load_dataset(self.DATASET_NAME, split="train", streaming=True)
35 |         tokenized_dataset = train_dataset.map(
36 |             self.tokenize_function,
37 |             batched=True,
38 |             num_proc=self.NUM_CPU,
39 |             remove_columns=["text"],
40 |         )
41 |         train_tokenized_dataset = tokenized_dataset.map(
42 |             self.group_texts,
43 |             batched=True,
44 |             num_proc=self.NUM_CPU,
45 |         )
46 |         train_tokenized_dataset.push_to_hub(self.HF_ACCOUNT_REPO)
47 | 
48 | if __name__ == '__main__':
49 |     parser = argparse.ArgumentParser(description="Process and push dataset to Hugging Face Hub")
50 |     parser.add_argument("--seed", type=int, default=42, help="Random seed")
51 |     parser.add_argument("--seq_len", type=int, default=8192, help="Sequence length for processing")
52 |     parser.add_argument("--hf_account", type=str, default="YOUR HUGGINGFACE API KEY", help="Hugging Face account name and repo")
53 |     parser.add_argument("--dataset_name", type=str, default="uggingFaceM4/VQAv2", help="Name of the dataset to process")
54 |     args = parser.parse_args()
55 |     dataset_builder = BuildDataset(seed=args.seed, seq_len=args.seq_len, hf_account=args.hf_account, dataset_name=args.dataset_name)
56 |     dataset_builder.build()


--------------------------------------------------------------------------------
/fts/inference/hf_model.py:
--------------------------------------------------------------------------------
 1 | import torch
 2 | import logging
 3 | from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
 4 | 
 5 | class Inference:
 6 |     def __init__(
 7 |             self, 
 8 |             model_id: str, 
 9 |             device: str = None, 
10 |             max_length: int = 20, 
11 |             quantize: bool = False, 
12 |             quantization_config: dict = None
13 |         ):
14 |         super().__init__()
15 |         self.logger = logging.getLogger(__name__)
16 |         self.device = device if device else ('cuda' if torch.cuda.is_available() else 'cpu')
17 |         self.model_id = model_id
18 |         self.max_length = max_length
19 | 
20 |         bnb_config = None
21 |         if quantize:
22 |             if not quantization_config:
23 |                 quantization_config = {
24 |                     'load_in_4bit': True,
25 |                     'bnb_4bit_use_double_quant': True,
26 |                     'bnb_4bit_quant_type': "nf4",
27 |                     'bnb_4bit_compute_dtype': torch.bfloat16
28 |                 }
29 |             bnb_config = BitsAndBytesConfig(**quantization_config)
30 | 
31 |         try:
32 |             self.tokenizer = AutoTokenizer.from_pretrained(self.model_id)
33 |             self.model = AutoModelForCausalLM.from_pretrained(self.model_id, quantization_config=bnb_config)
34 |             self.model.to(self.device)
35 |         except Exception as e:
36 |             self.logger.error(f"Failed to load the model or the tokenizer: {e}")
37 |             raise
38 | 
39 |     def __call__(self, prompt_text: str, max_length: int = None):
40 |         max_length = max_length if max_length else self.max_length
41 |         try:
42 |             inputs = self.tokenizer.encode(prompt_text, return_tensors="pt").to(self.device)
43 |             with torch.no_grad():
44 |                 outputs = self.model.generate(inputs, max_length=max_length, do_sample=True)
45 |             return self.tokenizer.decode(outputs[0], skip_special_tokens=True)
46 |         except Exception as e:
47 |             self.logger.error(f"Failed to generate the text: {e}")
48 |             raise
49 | 
50 | 
51 |     def run(self, prompt_text: str, max_length: int = None):
52 |         max_length = max_length if max_length else self.max_length
53 |         try:
54 |             inputs = self.tokenizer.encode(prompt_text, return_tensors="pt").to(self.device)
55 |             with torch.no_grad():
56 |                 outputs = self.model.generate(inputs, max_length=max_length, do_sample=True)
57 |             return self.tokenizer.decode(outputs[0], skip_special_tokens=True)
58 |         except Exception as e:
59 |             self.logger.error(f"Failed to generate the text: {e}")
60 |             raise
61 | 


--------------------------------------------------------------------------------
/mkdocs.yml:
--------------------------------------------------------------------------------
 1 | site_name: Finetuning Suite Docs
 2 | site_url: https://ft.apac.ai
 3 | site_author: APAC AI
 4 | site_description: Finetune any model with unparalled performance, speed, and reliability using Qlora, BNB, Lora, Peft in less than 30 seconds, just press GO.
 5 | repo_name: kyegomez/finetuning-suite
 6 | repo_url: https://github.com/kyegomez/finetuning-suite
 7 | edit_uri: https://github.com/kyegomez/finetuning-suite/tree/main/docs
 8 | copyright: APAC Corp 2023. All rights reserved.
 9 | 
10 | plugins:
11 |   - glightbox
12 |   - search
13 | copyright: "&copy; APAC Corp, Inc."
14 | extra_css:
15 |   - docs/assets/css/extra.css
16 | extra:
17 |   # analytics:
18 |   #   provider: google
19 |   #   property: G-QM8EDPSCB6
20 |   social:
21 |     - icon: fontawesome/solid/house
22 |       link: assets/img/ft-logo.png
23 |     - icon: fontawesome/brands/discord
24 |       link: https://discord.gg/qUtxnK2NMf
25 |     - icon: fontawesome/brands/github
26 |       link: https://github.com/kyegomez/finetuning-suite/
27 |     - icon: fontawesome/brands/python
28 |       link: https://pypi.org/project/finetuning-suite
29 | theme:
30 |     name: material
31 |     custom_dir: docs/overrides
32 |     logo: assets/img/ft-logo.png
33 |     palette:
34 |       # Palette toggle for light mode
35 |     - scheme: default
36 |       primary: 'custom'
37 |       toggle:
38 |         icon: material/brightness-7 
39 |         name: Switch to dark mode
40 |     # Palette toggle for dark mode
41 |     - scheme: slate
42 |       primary: 'custom'
43 |       accent: light blue
44 |       toggle:
45 |         icon: material/brightness-4
46 |         name: Switch to light mode
47 |     features:
48 |         - content.code.copy
49 |         - content.code.annotate
50 |         - navigation.tabs
51 |         - navigation.sections
52 |         - navigation.expand
53 |         - navigation.top
54 |         - announce.dismiss
55 |     font:
56 |       text: Roboto
57 |       code: Roboto Mono
58 | 
59 | extra_css:
60 |   - stylesheets/extra.css
61 | 
62 | markdown_extensions:
63 |   - pymdownx.highlight:
64 |       anchor_linenums: true
65 |       line_spans: __span
66 |       pygments_lang_class: true
67 |   - admonition
68 |   - pymdownx.inlinehilite
69 |   - pymdownx.snippets
70 |   - pymdownx.superfences
71 |   - pymdownx.details
72 |   - pymdownx.tabbed
73 |   - tables
74 |   - def_list
75 |   - footnotes
76 | 
77 | 
78 | nav:
79 | - Home:
80 |     - Overview: "index.md"
81 | - ft:
82 |     - Overview: "ft/index.md"
83 |     - ft:
84 |       - ft.FineTuner: "ft/finetuner.md"
85 |       - ft.Inference: "ft/inference.md"
86 |       - ft.GPTQInference: "ft/gptq_inference.md"
87 | - Examples:
88 |     - Overview: "examples/index.md"
89 |     - FlashAttention: "examples/nn/attentions/flash.md"
90 |     


--------------------------------------------------------------------------------
/docs/applications/customer_support.md:
--------------------------------------------------------------------------------
 1 | ## **Applications of Zeta: Revolutionizing Customer Support**
 2 | 
 3 | ---
 4 | 
 5 | **Introduction**:  
 6 | In today's fast-paced digital world, responsive and efficient customer support is a linchpin for business success. The introduction of AI-driven zeta in the customer support domain can transform the way businesses interact with and assist their customers. By leveraging the combined power of multiple AI agents working in concert, businesses can achieve unprecedented levels of efficiency, customer satisfaction, and operational cost savings.
 7 | 
 8 | ---
 9 | 
10 | ### **The Benefits of Using Zeta for Customer Support:**
11 | 
12 | 1. **24/7 Availability**: Zeta never sleep. Customers receive instantaneous support at any hour, ensuring constant satisfaction and loyalty.
13 |   
14 | 2. **Infinite Scalability**: Whether it's ten inquiries or ten thousand, zeta can handle fluctuating volumes with ease, eliminating the need for vast human teams and minimizing response times.
15 |   
16 | 3. **Adaptive Intelligence**: Zeta learn collectively, meaning that a solution found for one customer can be instantly applied to benefit all. This leads to constantly improving support experiences, evolving with every interaction.
17 | 
18 | ---
19 | 
20 | ### **Features - Reinventing Customer Support**:
21 | 
22 | - **AI Inbox Monitor**: Continuously scans email inboxes, identifying and categorizing support requests for swift responses.
23 |   
24 | - **Intelligent Debugging**: Proactively helps customers by diagnosing and troubleshooting underlying issues.
25 |   
26 | - **Automated Refunds & Coupons**: Seamless integration with payment systems like Stripe allows for instant issuance of refunds or coupons if a problem remains unresolved.
27 |   
28 | - **Full System Integration**: Holistically connects with CRM, email systems, and payment portals, ensuring a cohesive and unified support experience.
29 |   
30 | - **Conversational Excellence**: With advanced LLMs (Language Model Transformers), the swarm agents can engage in natural, human-like conversations, enhancing customer comfort and trust.
31 |   
32 | - **Rule-based Operation**: By working with rule engines, zeta ensure that all actions adhere to company guidelines, ensuring consistent, error-free support.
33 |   
34 | - **Turing Test Ready**: Crafted to meet and exceed the Turing Test standards, ensuring that every customer interaction feels genuine and personal.
35 | 
36 | ---
37 | 
38 | **Conclusion**:  
39 | Zeta are not just another technological advancement; they represent the future of customer support. Their ability to provide round-the-clock, scalable, and continuously improving support can redefine customer experience standards. By adopting zeta, businesses can stay ahead of the curve, ensuring unparalleled customer loyalty and satisfaction.
40 | 
41 | **Experience the future of customer support. Dive into the swarm revolution.**
42 | 
43 | 


--------------------------------------------------------------------------------
/docs/hiring.md:
--------------------------------------------------------------------------------
 1 | ## **Join the Swarm Revolution: Advancing Humanity & Prosperity Together!**
 2 | 
 3 | ### **The Next Chapter of Humanity's Story Begins Here...**
 4 | 
 5 | At Zeta, our mission transcends mere technological advancement. We envision a world where every individual can leverage the power of AI to uplift their lives, communities, and our shared future. If you are driven by the passion to revolutionize industries, to scale the heights of innovation, and believe in earning your fair share for every ounce of your dedication – you might be the one we're looking for.
 6 | 
 7 | ---
 8 | 
 9 | ### **Why Zeta?** 
10 | 
11 | #### **For the Ambitious Spirit**:
12 | - **Opportunity Beyond Boundaries**: Just as Fuller believed in the infinite opportunities of America, we believe in the limitless potential of raw Humantiy.
13 |   
14 | #### **For the Maverick**:
15 | - **Unprecedented Independence**: Like the Fuller salesmen, our team members have the autonomy to sculpt their roles, timelines, and outcomes. Here, you’re the captain of your ship.
16 | 
17 | #### **For the Avid Learner**:
18 | - **Continuous Learning & Growth**: Dive deep into the realms of AI, distributed systems, and customer success methodologies. We offer training, mentorship, and a platform to sharpen your skills.
19 | 
20 | #### **For the High Achiever**:
21 | - **Rewarding Compensation**: While the sky is the limit for your innovations, so is your earning potential. Prosper with performance-based rewards that reflect your dedication.
22 | 
23 | #### **For the Community Builder**:
24 | - **Culture of Unity & Innovation**: At Zeta, you’re not just an employee; you’re a pivotal part of our mission. Experience camaraderie, collaboration, and a shared purpose that binds us together.
25 | 
26 | #### **For the Visionary**:
27 | - **Work on the Cutting-Edge**: Be at the forefront of AI and technology. Shape solutions that will define the next era of human history.
28 | 
29 | ---
30 | 
31 | ### **Benefits of Joining Zeta**:
32 | 
33 | 1. **Advance Humanity**: Play an instrumental role in democratizing technology for all.
34 | 2. **Financial Prosperity**: Harness a compensation structure that grows with your achievements.
35 | 3. **Flexible Work Environment**: Customize your workspace, schedule, and workstyle.
36 | 4. **Global Network**: Collaborate with some of the brightest minds spanning continents.
37 | 5. **Personal Development**: Regular workshops, courses, and seminars to fuel your growth.
38 | 6. **Health & Wellness**: Comprehensive health benefits and well-being programs.
39 | 7. **Ownership & Equity**: As we grow, so does your stake and impact in our organization.
40 | 8. **Retreats & Team Building**: Forge bonds beyond work in exotic locations globally.
41 | 9. **Customer Success Impact**: Directly experience the joy of solving real-world challenges for our users.
42 | 
43 | ---
44 | 
45 | ### **Positions Open**:
46 | 
47 | - **AI & Swarm Engineers**: Architect, design, and optimize the swarm systems powering global innovations.
48 | 
49 | ---
50 | 
51 | ### **Your Invitation to the Future**:
52 | If you resonate with our vision of blending technological marvels with human brilliance, of creating a prosperous world where every dream has the wings of AI – we invite you to join us on this extraordinary journey.
53 | 
54 | **Are you ready to create history with Zeta?**
55 | 
56 | ---
57 | 
58 | **Apply Now and Let’s Push Our People Further!**
59 | 
60 | ---


--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
  1 | # Byte-compiled / optimized / DLL files
  2 | __pycache__/
  3 | *.py[cod]
  4 | *$py.class
  5 | 
  6 | # C extensions
  7 | *.so
  8 | 
  9 | # Distribution / packaging
 10 | .Python
 11 | build/
 12 | develop-eggs/
 13 | dist/
 14 | downloads/
 15 | eggs/
 16 | .eggs/
 17 | lib/
 18 | lib64/
 19 | parts/
 20 | .ruff_cache/
 21 | sdist/
 22 | var/
 23 | wheels/
 24 | share/python-wheels/
 25 | *.egg-info/
 26 | .installed.cfg
 27 | *.egg
 28 | MANIFEST
 29 | 
 30 | # PyInstaller
 31 | #  Usually these files are written by a python script from a template
 32 | #  before PyInstaller builds the exe, so as to inject date/other infos into it.
 33 | *.manifest
 34 | *.spec
 35 | 
 36 | # Installer logs
 37 | pip-log.txt
 38 | pip-delete-this-directory.txt
 39 | 
 40 | # Unit test / coverage reports
 41 | htmlcov/
 42 | .tox/
 43 | .nox/
 44 | .coverage
 45 | .coverage.*
 46 | .cache
 47 | nosetests.xml
 48 | coverage.xml
 49 | *.cover
 50 | *.py,cover
 51 | .hypothesis/
 52 | .pytest_cache/
 53 | cover/
 54 | 
 55 | # Translations
 56 | *.mo
 57 | *.pot
 58 | 
 59 | # Django stuff:
 60 | *.log
 61 | local_settings.py
 62 | db.sqlite3
 63 | db.sqlite3-journal
 64 | 
 65 | # Flask stuff:
 66 | instance/
 67 | .webassets-cache
 68 | 
 69 | # Scrapy stuff:
 70 | .scrapy
 71 | 
 72 | # Sphinx documentation
 73 | docs/_build/
 74 | 
 75 | # PyBuilder
 76 | .pybuilder/
 77 | target/
 78 | 
 79 | # Jupyter Notebook
 80 | .ipynb_checkpoints
 81 | 
 82 | # IPython
 83 | profile_default/
 84 | ipython_config.py
 85 | 
 86 | # pyenv
 87 | #   For a library or package, you might want to ignore these files since the code is
 88 | #   intended to run in multiple environments; otherwise, check them in:
 89 | # .python-version
 90 | 
 91 | # pipenv
 92 | #   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
 93 | #   However, in case of collaboration, if having platform-specific dependencies or dependencies
 94 | #   having no cross-platform support, pipenv may install dependencies that don't work, or not
 95 | #   install all needed dependencies.
 96 | #Pipfile.lock
 97 | 
 98 | # poetry
 99 | #   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
100 | #   This is especially recommended for binary packages to ensure reproducibility, and is more
101 | #   commonly ignored for libraries.
102 | #   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
103 | #poetry.lock
104 | 
105 | # pdm
106 | #   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
107 | #pdm.lock
108 | #   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
109 | #   in version control.
110 | #   https://pdm.fming.dev/#use-with-ide
111 | .pdm.toml
112 | 
113 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
114 | __pypackages__/
115 | 
116 | # Celery stuff
117 | celerybeat-schedule
118 | celerybeat.pid
119 | 
120 | # SageMath parsed files
121 | *.sage.py
122 | 
123 | # Environments
124 | .env
125 | .venv
126 | env/
127 | venv/
128 | ENV/
129 | env.bak/
130 | venv.bak/
131 | 
132 | # Spyder project settings
133 | .spyderproject
134 | .spyproject
135 | 
136 | # Rope project settings
137 | .ropeproject
138 | 
139 | # mkdocs documentation
140 | /site
141 | 
142 | # mypy
143 | .mypy_cache/
144 | .dmypy.json
145 | dmypy.json
146 | 
147 | # Pyre type checker
148 | .pyre/
149 | 
150 | # pytype static type analyzer
151 | .pytype/
152 | 
153 | # Cython debug symbols
154 | cython_debug/
155 | 
156 | # PyCharm
157 | #  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
158 | #  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
159 | #  and can be added to the global gitignore or merged into this file.  For a more nuclear
160 | #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
161 | #.idea/
162 | 


--------------------------------------------------------------------------------
/docs/applications/marketing_agencies.md:
--------------------------------------------------------------------------------
 1 | ## **Zeta in Marketing Agencies: A New Era of Automated Media Strategy**
 2 | 
 3 | ---
 4 | 
 5 | ### **Introduction**: 
 6 | - Brief background on marketing agencies and their role in driving brand narratives and sales.
 7 | - Current challenges and pain points faced in media planning, placements, and budgeting.
 8 | - Introduction to the transformative potential of zeta in reshaping the marketing industry.
 9 | 
10 | ---
11 | 
12 | ### **1. Fundamental Problem: Media Plan Creation**:
13 |    - **Definition**: The challenge of creating an effective media plan that resonates with a target audience and aligns with brand objectives.
14 |    
15 |    - **Traditional Solutions and Their Shortcomings**: Manual brainstorming sessions, over-reliance on past strategies, and long turnaround times leading to inefficiency.
16 |    
17 |    - **How Zeta Address This Problem**: 
18 |       - **Benefit 1**: Automated Media Plan Generation – Zeta ingest branding summaries, objectives, and marketing strategies to generate media plans, eliminating guesswork and human error.
19 |       - **Real-world Application of Zeta**: The automation of media plans based on client briefs, including platform selections, audience targeting, and creative versions.
20 | 
21 | ---
22 | 
23 | ### **2. Fundamental Problem: Media Placements**:
24 |    - **Definition**: The tedious task of determining where ads will be placed, considering demographics, platform specifics, and more.
25 |    
26 |    - **Traditional Solutions and Their Shortcomings**: Manual placement leading to possible misalignment with target audiences and brand objectives.
27 |    
28 |    - **How Zeta Address This Problem**: 
29 |       - **Benefit 2**: Precision Media Placements – Zeta analyze audience data and demographics to suggest the best placements, optimizing for conversions and brand reach.
30 |       - **Real-world Application of Zeta**: Automated selection of ad placements across platforms like Facebook, Google, and DSPs based on media plans.
31 | 
32 | ---
33 | 
34 | ### **3. Fundamental Problem: Budgeting**:
35 |    - **Definition**: Efficiently allocating and managing advertising budgets across multiple campaigns, platforms, and timeframes.
36 |    
37 |    - **Traditional Solutions and Their Shortcomings**: Manual budgeting using tools like Excel, prone to errors, and inefficient shifts in allocations.
38 |    
39 |    - **How Zeta Address This Problem**: 
40 |       - **Benefit 3**: Intelligent Media Budgeting – Zeta enable dynamic budget allocation based on performance analytics, maximizing ROI.
41 |       - **Real-world Application of Zeta**: Real-time adjustments in budget allocations based on campaign performance, eliminating long waiting periods and manual recalculations.
42 | 
43 | ---
44 | 
45 | ### **Features**:
46 | 1. Automated Media Plan Generator: Input your objectives and receive a comprehensive media plan.
47 | 2. Precision Media Placement Tool: Ensure your ads appear in the right places to the right people.
48 | 3. Dynamic Budget Allocation: Maximize ROI with real-time budget adjustments.
49 | 4. Integration with Common Tools: Seamless integration with tools like Excel and APIs for exporting placements.
50 | 5. Conversational Platform: A suite of tools built for modern marketing agencies, bringing all tasks under one umbrella.
51 | 
52 | ---
53 | 
54 | ### **Testimonials**:
55 | - "Zeta have completely revolutionized our media planning process. What used to take weeks now takes mere hours." - *Senior Media Strategist, Top-tier Marketing Agency*
56 | - "The precision with which we can place ads now is unprecedented. It's like having a crystal ball for marketing!" - *Campaign Manager, Global Advertising Firm*
57 | 
58 | ---
59 | 
60 | ### **Conclusion**: 
61 | - Reiterate the immense potential of zeta in revolutionizing media planning, placements, and budgeting for marketing agencies.
62 | - Call to action: For marketing agencies looking to step into the future and leave manual inefficiencies behind, zeta are the answer.
63 | 
64 | ---


--------------------------------------------------------------------------------
/docs/faq.md:
--------------------------------------------------------------------------------
 1 | **FAQ: Zeta - Crafting the Next Level in Neural Networks**
 2 | 
 3 | ---
 4 | 
 5 | We understand that delving into a new framework, especially in the ever-evolving world of machine learning, can be both exciting and a tad bit overwhelming. We've compiled some of the most frequently asked questions, hoping to bridge the gap between curiosity and clarity. You inspire us, and we want to ensure that your journey with Zeta is smooth and transformative.
 6 | 
 7 | ---
 8 | 
 9 | ## 1. How is Zeta different from PyTorch?
10 | 
11 | **Answer:** First and foremost, we have immense respect for PyTorch and the revolution it has brought to deep learning. However, Zeta is not just another deep learning framework. While PyTorch offers a robust platform for building neural networks from scratch, Zeta aims to make the process of creating State of The Art Models even more effortless and intuitive. 
12 | 
13 | - **Modularity**: Zeta's architecture allows for easily interchangeable modules, making it a breeze for developers to plug and play with different configurations.
14 |   
15 | - **LLMs & Multi-Modality**: We've integrated tools to efficiently harness the power of LLMs and Multi-Modal Foundation Models. This is not just about building a model; it's about building models that can interact, perceive, and reason with diverse data types - be it text, image, or more.
16 |   
17 | - **Enhanced Security and Trust**: Zeta enforces trust boundaries, schema validation, and provides tool activity-level permissions. This ensures that while your models are smart, they're also safe and adhere to set protocols.
18 | 
19 | - **Ease of Use**: Ever felt like going for a serene swim? Using Zeta feels just like that – fluid, intuitive, and without friction. Our pythonic methods, classes, and top-notch error handling guide you every step of the way.
20 | 
21 | - **Performance**: Think of Zeta as the Lamborghini of ML frameworks. It's built for speed, efficiency, and performance. Every single FLOP is put to its best use, ensuring swift model training and inference.
22 | 
23 | In essence, while PyTorch provides the building blocks, Zeta offers a refined, faster, and more intuitive experience to craft and deploy powerful neural networks.
24 | 
25 | ---
26 | 
27 | ## 2. How steep is the learning curve for Zeta, especially for someone accustomed to PyTorch?
28 | 
29 | **Answer:** We designed Zeta keeping both beginners and professionals in mind. If you're familiar with PyTorch, you'll appreciate the similarities in terms of syntax and structure. The added features and modules in Zeta are introduced with clarity and simplicity. With our comprehensive documentation, hands-on examples, and supportive community on [Discord](https://discord.gg/gnWRz88eym), we aim to make your transition smooth and enjoyable.
30 | 
31 | ---
32 | 
33 | ## 3. How does Zeta handle backward compatibility?
34 | 
35 | **Answer:** We understand the importance of backward compatibility, especially when developers invest time and resources into a framework. While we continually strive to innovate and introduce new features, we make sure that changes don't break the functionality of models built on earlier versions. We're committed to ensuring a balance between innovation and stability.
36 | 
37 | ---
38 | 
39 | ## 4. Are there plans for introducing more pre-trained models in Zeta?
40 | 
41 | **Answer:** Absolutely! Our vision with Zeta is not static. We are in the constant pursuit of integrating newer, state-of-the-art pre-trained models. Our goal is to give developers the arsenal they need to break new grounds in machine learning. Stay tuned for more exciting updates!
42 | 
43 | ---
44 | 
45 | ## 5. I'm facing a challenge with Zeta. How can I get help?
46 | 
47 | **Answer:** We're genuinely sorry to hear that, but rest assured, we're here to assist. Our [Discord community](https://discord.gg/gnWRz88eym) is active, and our team, along with fellow developers, are always eager to help. You can also raise an issue or start a discussion on our [Github Page](https://github.com/kyegomez). Remember, challenges are stepping stones to mastery, and we're with you every step of the way.
48 | 
49 | ---
50 | 
51 | Your feedback, questions, and concerns are the winds beneath our wings. Keep them coming, and together, let's shape the future of neural networks with Zeta.


--------------------------------------------------------------------------------
/fts/finetuner.py:
--------------------------------------------------------------------------------
  1 | import logging
  2 | 
  3 | import torch
  4 | from datasets import load_dataset
  5 | from peft import TaskType
  6 | from transformers import (
  7 |     AutoModelForCausalLM,
  8 |     AutoTokenizer,
  9 |     BitsAndBytesConfig,
 10 |     Seq2SeqTrainer,
 11 | )
 12 | 
 13 | from fts.inference.base import DefaultInferenceHandler
 14 | from fts.processing.base import DefaultPreprocessor
 15 | from fts.trainer.base import DefaultTrainerConfig
 16 | 
 17 | 
 18 | class FineTuner:
 19 |     def __init__(self, 
 20 |             model_id: str, 
 21 |             device: str = None,
 22 |             dataset_name=None, 
 23 |             lora_r=16,
 24 |             lora_alpha=32,
 25 |             lora_target_modules=["q", "v"],
 26 |             lora_bias="none",
 27 |             preprocessor=None,
 28 |             lora_task_type=TaskType.SEQ_2_SEQ_LM,
 29 |             max_length=1000, 
 30 |             quantize: bool = False, 
 31 |             quantization_config: dict = None,
 32 |             trainer_config=None,
 33 |             inference_handler=None
 34 |         ):
 35 |         self.logger = logging.getLogger(__name__)
 36 |         self.device = device if device else ('cuda' if torch.cuda.is_available() else 'cpu')
 37 |         self.model_id = model_id
 38 |         self.max_length = max_length
 39 |         self.dataset_name = dataset_name
 40 | 
 41 |         self.preprocessor = preprocessor if preprocessor else DefaultPreprocessor(self.model_id)
 42 |         self.trainer_config = trainer_config if trainer_config else DefaultTrainerConfig
 43 |         self.inference_handler = inference_handler if inference_handler else DefaultInferenceHandler()
 44 | 
 45 |         self.lora_r = lora_r
 46 |         self.lora_alpha = lora_alpha
 47 |         self.lora_target_modules = lora_target_modules
 48 |         self.lora_bias = lora_bias
 49 |         self.lora_task_type = lora_task_type
 50 | 
 51 | 
 52 |         self.dataset = load_dataset(dataset_name)
 53 |         self.tokenizer = AutoTokenizer.from_pretrained(self.model_id)
 54 | 
 55 |         bnb_config = None
 56 |         if quantize:
 57 |             if not quantization_config:
 58 |                 quantization_config = {
 59 |                     'load_in_4bit': True,
 60 |                     'bnb_4bit_use_double_quant': True,
 61 |                     'bnb_4bit_quant_type': "nf4",
 62 |                     'bnb_4bit_compute_dtype': torch.bfloat16
 63 |                 }
 64 |             bnb_config = BitsAndBytesConfig(**quantization_config)
 65 | 
 66 |         try:
 67 |             self.tokenizer = AutoTokenizer.from_pretrained(self.model_id)
 68 |             self.model = AutoModelForCausalLM.from_pretrained(self.model_id, quantization_config=bnb_config)
 69 |             self.model.to(self.device)
 70 |         except Exception as e:
 71 |             self.logger.error(f"Failed to load the model or the tokenizer: {e}")
 72 |             raise
 73 | 
 74 |     def __call__(self, prompt_text: str, max_length: int = None):
 75 |         max_length = max_length if max_length else self.max_length
 76 |         try:
 77 |             inputs = self.tokenizer.encode(prompt_text, return_tensors="pt").to(self.device)
 78 |             with torch.no_grad():
 79 |                 outputs = self.model.generate(inputs, max_length=max_length, do_sample=True)
 80 |             return self.tokenizer.decode(outputs[0], skip_special_tokens=True)
 81 |         except Exception as e:
 82 |             self.logger.error(f"Failed to generate the text: {e}")
 83 |             raise
 84 | 
 85 |     def preprocess_data(self):
 86 |         tokenized_dataset = self.dataset.map(self.preprocessor.preprocess_function, batched=True, remove_columns=["dialogue", "summary", "id"])
 87 |         return tokenized_dataset
 88 |     
 89 |     def train(self, output_dir, num_train_epochs):
 90 |         self.model, data_collator, training_args = self.trainer_config.configure(self.model, self.tokenizer, output_dir, num_train_epochs)
 91 |         
 92 |         tokenized_dataset = self.preprocessor_datas()
 93 |         trainer = Seq2SeqTrainer(model=self.model, args=training_args, data_collator=data_collator, train_dataset=tokenized_dataset["train"])
 94 |         trainer.train()
 95 | 
 96 |     def generate(self, prompt_text: str, max_length: int = None):
 97 |         try:
 98 |             return self.inference_handler.generate(prompt_text, self.model, self.tokenizer, self.device, max_length)
 99 |         except Exception as error:
100 |             error_msg = f"Failed to generate text for input: {prompt_text} because of Error {error} try modifying the inference function"
101 |             self.logger.error(error_msg)
102 |             raise ValueError(error_msg) from error
103 | 
104 | 
105 | 
106 | 
107 | 
108 | 
109 | 


--------------------------------------------------------------------------------
/docs/ft/gptq_inference.md:
--------------------------------------------------------------------------------
  1 | # Documentation for the GPTQInference Class
  2 | 
  3 | ## Introduction
  4 | 
  5 | `GPTQInference` is a class designed to leverage the capabilities of pre-trained GPT-like models from the HuggingFace transformers library, while also incorporating quantization for efficient inference. Quantization reduces the model size and inference time by using a smaller number of bits to represent the weights, which can be especially beneficial for deployment on resource-constrained devices.
  6 | 
  7 | ## Class Definition
  8 | 
  9 | ```python
 10 | class GPTQInference:
 11 |     def __init__(
 12 |         self,
 13 |         model_id: str,
 14 |         quantization_config_bits: int = 4,
 15 |         quantization_config_dataset: str = None,
 16 |         max_length: int = 500
 17 |     ):
 18 | ```
 19 | 
 20 | ### Parameters:
 21 | 
 22 | - `model_id (str)`: Identifier for the pre-trained model to be loaded. This typically corresponds to model names or paths in the HuggingFace Model Hub.
 23 |   
 24 | - `quantization_config_bits (int, default=4)`: Number of bits used for quantization. By default, it uses 4 bits.
 25 |   
 26 | - `quantization_config_dataset (str, default=None)`: Dataset identifier for the quantization process. If provided, the dataset is used to fine-tune the quantization parameters.
 27 |   
 28 | - `max_length (int, default=500)`: The maximum length of the generated sequences.
 29 | 
 30 | ## Functionality and Usage
 31 | 
 32 | ### Initialization
 33 | 
 34 | Upon instantiation, the class:
 35 | 
 36 | 1. Loads the tokenizer corresponding to the provided model identifier.
 37 | 2. Initializes a quantization configuration with the given parameters and the loaded tokenizer.
 38 | 3. Loads the model for causal language modeling based on the provided model identifier and attaches the quantization configuration to it.
 39 | 
 40 | ### Generation
 41 | 
 42 | The `generate` method provides a way to produce text based on a given prompt:
 43 | 
 44 | ```python
 45 |     def generate(self, prompt: str):
 46 | ```
 47 | 
 48 | #### Parameters:
 49 | 
 50 | - `prompt (str)`: The input string based on which the model will generate a continuation or completion.
 51 | 
 52 | #### Returns:
 53 | 
 54 | - `str`: The generated text continuation.
 55 | 
 56 | ### How It Works:
 57 | 
 58 | 1. The prompt is tokenized using the loaded tokenizer and converted into a tensor.
 59 | 2. The tensor is then passed to the model's `generate` method to produce a sequence of token IDs.
 60 | 3. The generated token IDs are decoded to produce the final text.
 61 | 
 62 | Note: In the case of any exceptions during the generation, an error message is printed and the exception is raised.
 63 | 
 64 | ## Usage Examples
 65 | 
 66 | ### Example 1: Basic Usage
 67 | 
 68 | ```python
 69 | from zeta import GPTQInference
 70 | 
 71 | model_id = "gpt2-medium"
 72 | inference_engine = GPTQInference(model_id)
 73 | output_text = inference_engine.generate("Once upon a time")
 74 | print(output_text)
 75 | ```
 76 | 
 77 | ### Example 2: Using Custom Quantization Bits
 78 | 
 79 | ```python
 80 | from zeta import GPTQInference
 81 | 
 82 | model_id = "gpt2-medium"
 83 | inference_engine = GPTQInference(model_id, quantization_config_bits=2)
 84 | output_text = inference_engine.generate("The future of AI is")
 85 | print(output_text)
 86 | ```
 87 | 
 88 | ### Example 3: Specifying a Dataset for Quantization
 89 | 
 90 | ```python
 91 | from zeta import GPTQInference
 92 | 
 93 | model_id = "gpt2-medium"
 94 | inference_engine = GPTQInference(model_id, quantization_config_dataset="my_dataset")
 95 | output_text = inference_engine.generate("The beauty of nature is")
 96 | print(output_text)
 97 | ```
 98 | 
 99 | ## Mathematical Formulation
100 | 
101 | Quantization is a process that involves mapping a continuous or large set of values to a finite range. For a weight \( w \) in the neural network, the quantized weight \( w_q \) can be represented as:
102 | 
103 | \[ w_q = Q(w, B) \]
104 | 
105 | Where:
106 | - \( Q \) is the quantization function.
107 | - \( B \) represents the number of bits used for quantization, which in our case is given by `quantization_config_bits`.
108 | 
109 | This process ensures that the model size is reduced and the inference becomes faster, albeit at the potential cost of some loss in precision.
110 | 
111 | ## Additional Tips
112 | 
113 | - While quantization can speed up model inference and reduce model size, it may also result in a slight degradation of model performance. It's always a good practice to evaluate the quantized model's performance on a validation set.
114 |   
115 | - If you encounter unexpected errors during inference, ensure that the `model_id` provided corresponds to a valid pre-trained model in the HuggingFace Model Hub.
116 | 
117 | ## References and Resources
118 | 
119 | - [HuggingFace Transformers Library](https://huggingface.co/transformers/)
120 |   
121 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | [![Multi-Modality](images/agorabanner.png)](https://discord.gg/qUtxnK2NMf)
  2 | 
  3 | 
  4 | ![Finetuning suite logo](images/ft-logo.png)
  5 | 
  6 | Finetune any model with unparalled performance, speed, and reliability using Qlora, BNB, Lora, Peft in less than 30 seconds, just press GO.
  7 | 
  8 | 
  9 | # 🤝 Schedule a 1-on-1 Session
 10 | Book a [1-on-1 Session with Kye](https://calendly.com/apacai/agora), the Creator, to discuss any issues, provide feedback, or explore how we can improve Zeta for you.
 11 | 
 12 | ---
 13 | 
 14 | ## 📦 Installation 📦
 15 | 
 16 | ```bash
 17 | $ pip3 install ft-suite
 18 | ```
 19 | 
 20 | ---
 21 | ## 🚀 Quick Start 🚀
 22 | 
 23 | ### Finetuning
 24 | 
 25 | ```python
 26 | from fts import FineTuner
 27 | 
 28 | # Initialize the fine tuner
 29 | model_id="google/flan-t5-xxl"
 30 | dataset_name = "samsung"
 31 | 
 32 | tuner = FineTuner(
 33 |     model_id=model_id,
 34 |     dataset_name=dataset_name,
 35 |     max_length=150,
 36 |     lora_r=16,
 37 |     lora_alpha=32,
 38 |     quantize=True
 39 | )
 40 | 
 41 | # Generate content
 42 | prompt_text = "Summarize this idea for me."
 43 | print(tuner(prompt_text))
 44 | ```
 45 | 
 46 | ----
 47 | 
 48 | ## Inference
 49 | ```python
 50 | from fts import Inference
 51 | 
 52 | model = Inference(
 53 |     model_id="georgesung/llama2_7b_chat_uncensored",
 54 |     quantized=True
 55 | )
 56 | 
 57 | model.run("What is your name")
 58 | ```
 59 | 
 60 | 
 61 | ## GPTQ Inference
 62 | 
 63 | ```python
 64 | 
 65 | from fts import GPTQInference
 66 | 
 67 | 
 68 | model_id = "facebook/opt-125m"
 69 | model = GPTQInference(model_id=model_id, max_length=400)
 70 | 
 71 | prompt = "in a land far far away"
 72 | result = model.run(prompt)
 73 | print(result)
 74 | 
 75 | ```
 76 | 
 77 | ---
 78 | 
 79 | ## 🎉 Features 🎉
 80 | 
 81 | - **World-Class Quantization**: Get the most out of your models with top-tier performance and preserved accuracy! 🏋️‍♂️
 82 |   
 83 | - **Automated PEFT**: Simplify your workflow! Let our toolkit handle the optimizations. 🛠️
 84 | 
 85 | - **LoRA Configuration**: Dive into the potential of flexible LoRA configurations, a game-changer for performance! 🌌
 86 | 
 87 | - **Seamless Integration**: Designed to work seamlessly with popular models like LLAMA, Falcon, and more! 🤖
 88 | 
 89 | 
 90 | ----
 91 | 
 92 | ## 🛣️ Roadmap 🛣️
 93 | 
 94 | Here's a sneak peek into our ambitious roadmap! We're always evolving, and your feedback and contributions can shape our journey! ✨
 95 | 
 96 | - [ ] **More Example Scripts**:
 97 |   - [ ] Using GPT models
 98 |   - [ ] Transfer learning examples
 99 |   - [ ] Real-world application samples
100 | 
101 | - [ ] **Polymorphic Preprocessing Function**:
102 |   - [ ] Design a function to handle diverse datasets
103 |   - [ ] Integrate with known dataset structures from popular sources
104 |   - [ ] Custom dataset blueprint for user-defined structures
105 | 
106 | - [ ] **Extended Model Support**:
107 |   - [ ] Integration with Lama, Falcon, etc.
108 |   - [ ] Support for non-English models
109 | 
110 | - [ ] **Comprehensive Documentation**:
111 |   - [ ] Detailed usage guide
112 |   - [ ] Best practices for fine-tuning
113 |   - [ ] Benchmarks for quantization and LoRA features
114 |   
115 | - [ ] **Interactive Web Interface**:
116 |   - [ ] GUI for easy fine-tuning
117 |   - [ ] Visualization tools for model insights
118 | 
119 | - [ ] **Advanced Features**:
120 |   - [ ] Integration with other quantization techniques
121 |   - [ ] Support for more task types beyond text generation
122 |   - [ ] Model debugging and introspection tools
123 |   - [ ] Integrate TRLX from Carper
124 | 
125 | ... And so much more coming up!
126 | 
127 | -----
128 | 
129 | ## 💌 Feedback & Contributions 💌
130 | 
131 | We're excited about the journey ahead and would love to have you with us! For feedback, suggestions, or contributions, feel free to open an issue or a pull request. Let's shape the future of fine-tuning together! 🌱
132 | 
133 | ----
134 | 
135 | ## 📜 License 📜
136 | 
137 | MIT
138 | 
139 | ---
140 | 
141 | # Share the Love! 💙
142 | 
143 | Spread the message of the Finetuning-Suite, this is an foundational tool to help everyone quantize and finetune state of the art models.
144 | 
145 | Sharing the project helps us reach more people who could benefit from it, and it motivates us to continue developing and improving the suite.
146 | 
147 | Click the buttons below to share Finetuning-Suite on your favorite social media platforms:
148 | 
149 | - [Share on Twitter](https://twitter.com/intent/tweet?url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite&text=Check%20out%20Finetuning-Suite!%20A%20great%20resource%20for%20machine%20learning%20finetuning.%20%23AI%20%23MachineLearning%20%23GitHub)
150 | 
151 | - [Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite)
152 | 
153 | - [Share on LinkedIn](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite&title=Finetuning-Suite&summary=Check%20out%20this%20fantastic%20resource%20for%20machine%20learning%20finetuning!)
154 | 
155 | - [Share on Reddit](https://reddit.com/submit?url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite&title=Check%20out%20Finetuning-Suite!)
156 | 
157 | Also, we'd love to see how you're using Finetuning-Suite! Share your projects and experiences with us by tagging us on Twitter [@finetuning-suite](https://twitter.com/kyegomezb).
158 | 
159 | Lastly, don't forget to ⭐️ the repository if you find it useful. Your support means a lot to us! Thank you! 💙
160 | 
161 | 
162 | ----
163 | 
164 | 
165 | 
166 | 


--------------------------------------------------------------------------------
/docs/ft/index.md:
--------------------------------------------------------------------------------
  1 | [![Multi-Modality](images/agorabanner.png)](https://discord.gg/qUtxnK2NMf)
  2 | 
  3 | 
  4 | ![Finetuning suite logo](images/ft-logo.png)
  5 | 
  6 | Finetune any model with unparalled performance, speed, and reliability using Qlora, BNB, Lora, Peft in less than 30 seconds, just press GO.
  7 | 
  8 | 
  9 | # 🤝 Schedule a 1-on-1 Session
 10 | Book a [1-on-1 Session with Kye](https://calendly.com/apacai/agora), the Creator, to discuss any issues, provide feedback, or explore how we can improve Zeta for you.
 11 | 
 12 | ---
 13 | 
 14 | ## 📦 Installation 📦
 15 | 
 16 | ```bash
 17 | $ pip3 install ft-suite
 18 | ```
 19 | 
 20 | ---
 21 | ## 🚀 Quick Start 🚀
 22 | 
 23 | ### Finetuning
 24 | 
 25 | ```python
 26 | from fts import FineTuner
 27 | 
 28 | # Initialize the fine tuner
 29 | model_id="google/flan-t5-xxl"
 30 | dataset_name = "samsung"
 31 | 
 32 | tuner = FineTuner(
 33 |     model_id=model_id,
 34 |     dataset_name=dataset_name,
 35 |     max_length=150,
 36 |     lora_r=16,
 37 |     lora_alpha=32,
 38 |     quantize=True
 39 | )
 40 | 
 41 | # Generate content
 42 | prompt_text = "Summarize this idea for me."
 43 | print(tuner(prompt_text))
 44 | ```
 45 | 
 46 | ----
 47 | 
 48 | ## Inference
 49 | ```python
 50 | from fts import Inference
 51 | 
 52 | model = Inference(
 53 |     model_id="georgesung/llama2_7b_chat_uncensored",
 54 |     quantized=True
 55 | )
 56 | 
 57 | model.run("What is your name")
 58 | ```
 59 | 
 60 | 
 61 | ## GPTQ Inference
 62 | 
 63 | ```python
 64 | 
 65 | from fts import GPTQInference
 66 | 
 67 | 
 68 | model_id = "facebook/opt-125m"
 69 | model = GPTQInference(model_id=model_id, max_length=400)
 70 | 
 71 | prompt = "in a land far far away"
 72 | result = model.run(prompt)
 73 | print(result)
 74 | 
 75 | ```
 76 | 
 77 | ---
 78 | 
 79 | ## 🎉 Features 🎉
 80 | 
 81 | - **World-Class Quantization**: Get the most out of your models with top-tier performance and preserved accuracy! 🏋️‍♂️
 82 |   
 83 | - **Automated PEFT**: Simplify your workflow! Let our toolkit handle the optimizations. 🛠️
 84 | 
 85 | - **LoRA Configuration**: Dive into the potential of flexible LoRA configurations, a game-changer for performance! 🌌
 86 | 
 87 | - **Seamless Integration**: Designed to work seamlessly with popular models like LLAMA, Falcon, and more! 🤖
 88 | 
 89 | 
 90 | ----
 91 | 
 92 | ## 🛣️ Roadmap 🛣️
 93 | 
 94 | Here's a sneak peek into our ambitious roadmap! We're always evolving, and your feedback and contributions can shape our journey! ✨
 95 | 
 96 | - [ ] **More Example Scripts**:
 97 |   - [ ] Using GPT models
 98 |   - [ ] Transfer learning examples
 99 |   - [ ] Real-world application samples
100 | 
101 | - [ ] **Polymorphic Preprocessing Function**:
102 |   - [ ] Design a function to handle diverse datasets
103 |   - [ ] Integrate with known dataset structures from popular sources
104 |   - [ ] Custom dataset blueprint for user-defined structures
105 | 
106 | - [ ] **Extended Model Support**:
107 |   - [ ] Integration with Lama, Falcon, etc.
108 |   - [ ] Support for non-English models
109 | 
110 | - [ ] **Comprehensive Documentation**:
111 |   - [ ] Detailed usage guide
112 |   - [ ] Best practices for fine-tuning
113 |   - [ ] Benchmarks for quantization and LoRA features
114 |   
115 | - [ ] **Interactive Web Interface**:
116 |   - [ ] GUI for easy fine-tuning
117 |   - [ ] Visualization tools for model insights
118 | 
119 | - [ ] **Advanced Features**:
120 |   - [ ] Integration with other quantization techniques
121 |   - [ ] Support for more task types beyond text generation
122 |   - [ ] Model debugging and introspection tools
123 |   - [ ] Integrate TRLX from Carper
124 | 
125 | ... And so much more coming up!
126 | 
127 | -----
128 | 
129 | ## 💌 Feedback & Contributions 💌
130 | 
131 | We're excited about the journey ahead and would love to have you with us! For feedback, suggestions, or contributions, feel free to open an issue or a pull request. Let's shape the future of fine-tuning together! 🌱
132 | 
133 | ----
134 | 
135 | ## 📜 License 📜
136 | 
137 | MIT
138 | 
139 | ---
140 | 
141 | # Share the Love! 💙
142 | 
143 | Spread the message of the Finetuning-Suite, this is an foundational tool to help everyone quantize and finetune state of the art models.
144 | 
145 | Sharing the project helps us reach more people who could benefit from it, and it motivates us to continue developing and improving the suite.
146 | 
147 | Click the buttons below to share Finetuning-Suite on your favorite social media platforms:
148 | 
149 | - [Share on Twitter](https://twitter.com/intent/tweet?url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite&text=Check%20out%20Finetuning-Suite!%20A%20great%20resource%20for%20machine%20learning%20finetuning.%20%23AI%20%23MachineLearning%20%23GitHub)
150 | 
151 | - [Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite)
152 | 
153 | - [Share on LinkedIn](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite&title=Finetuning-Suite&summary=Check%20out%20this%20fantastic%20resource%20for%20machine%20learning%20finetuning!)
154 | 
155 | - [Share on Reddit](https://reddit.com/submit?url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FFinetuning-Suite&title=Check%20out%20Finetuning-Suite!)
156 | 
157 | Also, we'd love to see how you're using Finetuning-Suite! Share your projects and experiences with us by tagging us on Twitter [@finetuning-suite](https://twitter.com/kyegomezb).
158 | 
159 | Lastly, don't forget to ⭐️ the repository if you find it useful. Your support means a lot to us! Thank you! 💙
160 | 
161 | 
162 | ----
163 | 
164 | 
165 | 


--------------------------------------------------------------------------------
/docs/ft/inference.md:
--------------------------------------------------------------------------------
  1 | # Module Name: Inference
  2 | 
  3 | The `Inference` class is a part of a custom module that facilitates text generation using a pre-trained causal language model from the Hugging Face `transformers` library. The class provides functionalities for loading a pre-trained model, tokenizing input text, and generating text based on a given prompt. Additionally, it supports quantization of model weights to reduce the model size and accelerate inference.
  4 | 
  5 | ## Class Definition
  6 | 
  7 | ```python
  8 | class Inference:
  9 |     def __init__(
 10 |             self, 
 11 |             model_id: str, 
 12 |             device: str = None, 
 13 |             max_length: int = 20, 
 14 |             quantize: bool = False, 
 15 |             quantization_config: dict = None
 16 |         ):
 17 | ```
 18 | 
 19 | ### Parameters:
 20 | 
 21 | - `model_id` (str): The identifier of the pre-trained model to be loaded. This can be the path to a local directory containing the model files or a model id from the Hugging Face model hub.
 22 | - `device` (str, optional): The device on which the model will be loaded and inference will be performed. Default is `None`, which means that it will use CUDA if available, otherwise CPU.
 23 | - `max_length` (int, optional): The maximum length of the generated text. Default is 20.
 24 | - `quantize` (bool, optional): A flag indicating whether to quantize the model weights. Default is `False`.
 25 | - `quantization_config` (dict, optional): A dictionary containing the configuration for quantization. Default is `None`.
 26 | 
 27 | ## Methods
 28 | 
 29 | ### `__call__(self, prompt_text: str, max_length: int = None) -> str`
 30 | 
 31 | Generates text based on the provided `prompt_text`.
 32 | 
 33 | #### Parameters:
 34 | 
 35 | - `prompt_text` (str): The text prompt based on which the text will be generated.
 36 | - `max_length` (int, optional): The maximum length of the generated text. If not provided, the `max_length` specified during initialization will be used.
 37 | 
 38 | #### Returns:
 39 | 
 40 | - `str`: The generated text.
 41 | 
 42 | ### `run(self, prompt_text: str, max_length: int = None) -> str`
 43 | 
 44 | This method is an alternative to the `__call__` method and performs the same operation.
 45 | 
 46 | #### Parameters:
 47 | 
 48 | - `prompt_text` (str): The text prompt based on which the text will be generated.
 49 | - `max_length` (int, optional): The maximum length of the generated text. If not provided, the `max_length` specified during initialization will be used.
 50 | 
 51 | #### Returns:
 52 | 
 53 | - `str`: The generated text.
 54 | 
 55 | ## Usage Examples:
 56 | 
 57 | ### Example 1: Basic Usage
 58 | 
 59 | ```python
 60 | from finetuning_suite import Inference
 61 | 
 62 | model_id = "gpt2-small"
 63 | inference = Inference(model_id=model_id)
 64 | 
 65 | prompt_text = "Once upon a time"
 66 | generated_text = inference(prompt_text)
 67 | print(generated_text)
 68 | ```
 69 | 
 70 | ### Example 2: Specifying Maximum Length
 71 | 
 72 | ```python
 73 | from finetuning_suite import Inference
 74 | 
 75 | model_id = "gpt2-small"
 76 | inference = Inference(model_id=model_id, max_length=50)
 77 | 
 78 | prompt_text = "In a land far, far away"
 79 | generated_text = inference.run(prompt_text, max_length=30)
 80 | print(generated_text)
 81 | ```
 82 | 
 83 | ### Example 3: Using Quantization
 84 | 
 85 | ```python
 86 | from zeta import Inference
 87 | 
 88 | from finetuning_suite import Inference
 89 | quantization_config = {
 90 |     'load_in_4bit': True,
 91 |     'bnb_4bit_use_double_quant': True,
 92 |     'bnb_4bit_quant_type': "nf4",
 93 |     'bnb_4bit_compute_dtype': torch.bfloat16
 94 | }
 95 | inference = Inference(model_id=model_id, quantize=True, quantization_config=quantization_config)
 96 | 
 97 | prompt_text = "Once upon a time"
 98 | generated_text = inference(prompt_text)
 99 | print(generated_text)
100 | ```
101 | 
102 | ## Mathematical Formulation:
103 | 
104 | The `Inference` class uses a pre-trained causal language model for text generation. The probability of each word in the vocabulary is computed using the softmax function:
105 | 
106 | \[ P(w_i | w_1, ..., w_{i-1}) = \frac{e^{z_i}}{\sum_{j=1}^{V} e^{z_j}} \]
107 | 
108 | Where:
109 | - \( w_i \) is the ith word in the sequence.
110 | - \( z_i \) is the logit for the ith word in the vocabulary.
111 | - \( V \) is the size of the vocabulary.
112 | 
113 | The text is generated word by word, where each word is sampled from the probability distribution over the vocabulary computed by the model.
114 | 
115 | ## Limitations:
116 | 
117 | 1. Memory Consumption: Generating text with large models requires a significant amount of GPU memory. It is recommended to use a GPU with at least 16 GB of memory for generating text with large models.
118 | 
119 | 2. Computation Time: Generating text with large models requires a significant amount of computation time. It is recommended to use a powerful GPU to accelerate the inference process.
120 | 
121 | 3. Quantization Accuracy: Quantizing the model weights reduces the model size and accelerates inference, but may also result in a slight decrease in model accuracy. It is recommended to evaluate the quantized model on a validation set to ensure that the accuracy is acceptable for the specific application.
122 | 
123 | ## Conclusion:
124 | 
125 | The `Inference` class facilitates text generation using pre-trained models from the Hugging Face `transformers` library. This class includes functionalities for loading a pre-trained model, tokenizing input text, and generating text based on a given prompt. It also supports quantization of model weights to reduce the model size and accelerate inference.


--------------------------------------------------------------------------------
/docs/design.md:
--------------------------------------------------------------------------------
  1 | # Design Philosophy Document for Zeta
  2 | 
  3 | ## Usable
  4 | 
  5 | ### Objective
  6 | 
  7 | Our goal is to ensure that Zeta is intuitive and easy to use for all users, regardless of their level of technical expertise. This includes the developers who implement Zeta in their applications, as well as end users who interact with the implemented systems.
  8 | 
  9 | ### Tactics
 10 | 
 11 | - Clear and Comprehensive Documentation: We will provide well-written and easily accessible documentation that guides users through using and understanding Zeta.
 12 | - User-Friendly APIs: We'll design clean and self-explanatory APIs that help developers to understand their purpose quickly.
 13 | - Prompt and Effective Support: We will ensure that support is readily available to assist users when they encounter problems or need help with Zeta.
 14 | 
 15 | ## Reliable
 16 | 
 17 | ### Objective
 18 | 
 19 | Zeta should be dependable and trustworthy. Users should be able to count on Zeta to perform consistently and without error or failure.
 20 | 
 21 | ### Tactics
 22 | 
 23 | - Robust Error Handling: We will focus on error prevention, detection, and recovery to minimize failures in Zeta.
 24 | - Comprehensive Testing: We will apply various testing methodologies such as unit testing, integration testing, and stress testing to validate the reliability of our software.
 25 | - Continuous Integration/Continuous Delivery (CI/CD): We will use CI/CD pipelines to ensure that all changes are tested and validated before they're merged into the main branch.
 26 | 
 27 | ## Fast
 28 | 
 29 | ### Objective
 30 | 
 31 | Zeta should offer high performance and rapid response times. The system should be able to handle requests and tasks swiftly.
 32 | 
 33 | ### Tactics
 34 | 
 35 | - Efficient Algorithms: We will focus on optimizing our algorithms and data structures to ensure they run as quickly as possible.
 36 | - Caching: Where appropriate, we will use caching techniques to speed up response times.
 37 | - Profiling and Performance Monitoring: We will regularly analyze the performance of Zeta to identify bottlenecks and opportunities for improvement.
 38 | 
 39 | ## Scalable
 40 | 
 41 | ### Objective
 42 | 
 43 | Zeta should be able to grow in capacity and complexity without compromising performance or reliability. It should be able to handle increased workloads gracefully.
 44 | 
 45 | ### Tactics
 46 | 
 47 | - Modular Architecture: We will design Zeta using a modular architecture that allows for easy scaling and modification.
 48 | - Load Balancing: We will distribute tasks evenly across available resources to prevent overload and maximize throughput.
 49 | - Horizontal and Vertical Scaling: We will design Zeta to be capable of both horizontal (adding more machines) and vertical (adding more power to an existing machine) scaling.
 50 | 
 51 | ### Philosophy
 52 | 
 53 | Zeta is designed with a philosophy of simplicity and reliability. We believe that software should be a tool that empowers users, not a hurdle that they need to overcome. Therefore, our focus is on usability, reliability, speed, and scalability. We want our users to find Zeta intuitive and dependable, fast and adaptable to their needs. This philosophy guides all of our design and development decisions.
 54 | 
 55 | # Swarm Architecture Design Document
 56 | 
 57 | ## Overview
 58 | 
 59 | The goal of the Swarm Architecture is to provide a flexible and scalable system to build swarm intelligence models that can solve complex problems. This document details the proposed design to create a plug-and-play system, which makes it easy to create custom zeta, and provides pre-configured zeta with multi-modal agents.
 60 | 
 61 | ## Design Principles
 62 | 
 63 | - **Modularity**: The system will be built in a modular fashion, allowing various components to be easily swapped or upgraded.
 64 | - **Interoperability**: Different swarm classes and components should be able to work together seamlessly.
 65 | - **Scalability**: The design should support the growth of the system by adding more components or zeta.
 66 | - **Ease of Use**: Users should be able to easily create their own zeta or use pre-configured ones with minimal configuration.
 67 | 
 68 | ## Design Components
 69 | 
 70 | ### AbstractSwarm
 71 | 
 72 | The AbstractSwarm is an abstract base class which defines the basic structure of a swarm and the methods that need to be implemented. Any new swarm should inherit from this class and implement the required methods.
 73 | 
 74 | ### Swarm Classes
 75 | 
 76 | Various Swarm classes can be implemented inheriting from the AbstractSwarm class. Each swarm class should implement the required methods for initializing the components, worker nodes, and boss node, and running the swarm.
 77 | 
 78 | Pre-configured swarm classes with multi-modal agents can be provided for ease of use. These classes come with a default configuration of tools and agents, which can be used out of the box.
 79 | 
 80 | ### Tools and Agents
 81 | 
 82 | Tools and agents are the components that provide the actual functionality to the zeta. They can be language models, AI assistants, vector stores, or any other components that can help in problem solving.
 83 | 
 84 | To make the system plug-and-play, a standard interface should be defined for these components. Any new tool or agent should implement this interface, so that it can be easily plugged into the system.
 85 | 
 86 | ## Usage
 87 | 
 88 | Users can either use pre-configured zeta or create their own custom zeta.
 89 | 
 90 | To use a pre-configured swarm, they can simply instantiate the corresponding swarm class and call the run method with the required objective.
 91 | 
 92 | To create a custom swarm, they need to:
 93 | 
 94 | 1. Define a new swarm class inheriting from AbstractSwarm.
 95 | 2. Implement the required methods for the new swarm class.
 96 | 3. Instantiate the swarm class and call the run method.
 97 | 
 98 | ### Example
 99 | 
100 | ```python
101 | # Using pre-configured swarm
102 | swarm = PreConfiguredSwarm(openai_api_key)
103 | swarm.run_zeta(objective)
104 | 
105 | # Creating custom swarm
106 | class CustomSwarm(AbstractSwarm):
107 |     # Implement required methods
108 | 
109 | swarm = CustomSwarm(openai_api_key)
110 | swarm.run_zeta(objective)
111 | ```
112 | 


--------------------------------------------------------------------------------
/docs/contributing.md:
--------------------------------------------------------------------------------
  1 | # Contributing
  2 | 
  3 | Thank you for your interest in contributing to Zeta! We welcome contributions from the community to help improve usability and readability. By contributing, you can be a part of creating a dynamic and interactive AI system.
  4 | 
  5 | To get started, please follow the guidelines below.
  6 | 
  7 | 
  8 | ## Optimization Priorities
  9 | 
 10 | To continuously improve Zeta, we prioritize the following design objectives:
 11 | 
 12 | 1. **Usability**: Increase the ease of use and user-friendliness of the swarm system to facilitate adoption and interaction with basic input.
 13 | 
 14 | 2. **Reliability**: Improve the swarm's ability to obtain the desired output even with basic and un-detailed input.
 15 | 
 16 | 3. **Speed**: Reduce the time it takes for the swarm to accomplish tasks by improving the communication layer, critiquing, and self-alignment with meta prompting.
 17 | 
 18 | 4. **Scalability**: Ensure that the system is asynchronous, concurrent, and self-healing to support scalability.
 19 | 
 20 | Our goal is to continuously improve Zeta by following this roadmap while also being adaptable to new needs and opportunities as they arise.
 21 | 
 22 | ## Join the Zeta Community
 23 | 
 24 | Join the Zeta community on Discord to connect with other contributors, coordinate work, and receive support.
 25 | 
 26 | - [Join the Zeta Discord Server](https://discord.gg/qUtxnK2NMf)
 27 | 
 28 | 
 29 | ## Report and Issue
 30 | The easiest way to contribute to our docs is through our public [issue tracker](https://github.com/kyegomez/finetun/issues). Feel free to submit bugs, request features or changes, or contribute to the project directly. 
 31 | 
 32 | ## Pull Requests
 33 | 
 34 | Zeta docs are built using [MkDocs](https://squidfunk.github.io/mkdocs-material/getting-started/). 
 35 | 
 36 | To directly contribute to Zeta documentation, first fork the [zeta-docs](https://github.com/kyegomez/zeta) repository to your GitHub account. Then clone your repository to your local machine.
 37 | 
 38 | From inside the directory run: 
 39 | 
 40 | ```pip install -r requirements.txt```
 41 | 
 42 | To run `zeta-docs` locally run: 
 43 | 
 44 | ```mkdocs serve```
 45 | 
 46 | You should see something similar to the following: 
 47 | 
 48 | ```
 49 | INFO     -  Building documentation...
 50 | INFO     -  Cleaning site directory
 51 | INFO     -  Documentation built in 0.19 seconds
 52 | INFO     -  [09:28:33] Watching paths for changes: 'docs', 'mkdocs.yml'
 53 | INFO     -  [09:28:33] Serving on http://127.0.0.1:8000/
 54 | INFO     -  [09:28:37] Browser connected: http://127.0.0.1:8000/
 55 | ```
 56 | 
 57 | Follow the typical PR process to contribute changes. 
 58 | 
 59 | * Create a feature branch.
 60 | * Commit changes.
 61 | * Submit a PR.
 62 | 
 63 | 
 64 | -------
 65 | ---
 66 | 
 67 | ## Taking on Tasks
 68 | 
 69 | We have a growing list of tasks and issues that you can contribute to. To get started, follow these steps:
 70 | 
 71 | 1. Visit the [Zeta GitHub repository](https://github.com/kyegomez/zeta) and browse through the existing issues.
 72 | 
 73 | 2. Find an issue that interests you and make a comment stating that you would like to work on it. Include a brief description of how you plan to solve the problem and any questions you may have.
 74 | 
 75 | 3. Once a project coordinator assigns the issue to you, you can start working on it.
 76 | 
 77 | If you come across an issue that is unclear but still interests you, please post in the Discord server mentioned above. Someone from the community will be able to help clarify the issue in more detail.
 78 | 
 79 | We also welcome contributions to documentation, such as updating markdown files, adding docstrings, creating system architecture diagrams, and other related tasks.
 80 | 
 81 | ## Submitting Your Work
 82 | 
 83 | To contribute your changes to Zeta, please follow these steps:
 84 | 
 85 | 1. Fork the Zeta repository to your GitHub account. You can do this by clicking on the "Fork" button on the repository page.
 86 | 
 87 | 2. Clone the forked repository to your local machine using the `git clone` command.
 88 | 
 89 | 3. Before making any changes, make sure to sync your forked repository with the original repository to keep it up to date. You can do this by following the instructions [here](https://docs.github.com/en/github/collaborating-with-pull-requests/syncing-a-fork).
 90 | 
 91 | 4. Create a new branch for your changes. This branch should have a descriptive name that reflects the task or issue you are working on.
 92 | 
 93 | 5. Make your changes in the branch, focusing on a small, focused change that only affects a few files.
 94 | 
 95 | 6. Run any necessary formatting or linting tools to ensure that your changes adhere to the project's coding standards.
 96 | 
 97 | 7. Once your changes are ready, commit them to your branch with descriptive commit messages.
 98 | 
 99 | 8. Push the branch to your forked repository.
100 | 
101 | 9. Create a pull request (PR) from your branch to the main Zeta repository. Provide a clear and concise description of your changes in the PR.
102 | 
103 | 10. Request a review from the project maintainers. They will review your changes, provide feedback, and suggest any necessary improvements.
104 | 
105 | 11. Make any required updates or address any feedback provided during the review process.
106 | 
107 | 12. Once your changes have been reviewed and approved, they will be merged into the main branch of the Zeta repository.
108 | 
109 | 13. Congratulations! You have successfully contributed to Zeta.
110 | 
111 | Please note that during the review process, you may be asked to make changes or address certain issues. It is important to engage in open and constructive communication with the project maintainers to ensure the quality of your contributions.
112 | 
113 | ## Developer Setup
114 | 
115 | If you are interested in setting up the Zeta development environment, please follow the instructions provided in the [developer setup guide](docs/developer-setup.md). This guide provides an overview of the different tools and technologies used in the project.
116 | 
117 | ## Join the Agora Community
118 | 
119 | Zeta is brought to you by Agora, the open-source AI research organization. Join the Agora community to connect with other researchers and developers working on AI projects.
120 | 
121 | - [Join the Agora Discord Server](https://discord.gg/qUtxnK2NMf)
122 | 
123 | Thank you for your contributions and for being a part of the Zeta and Agora community! Together, we can advance Humanity through the power of AI.


--------------------------------------------------------------------------------
/docs/bounties.md:
--------------------------------------------------------------------------------
 1 | # Bounty Program
 2 | 
 3 | Our bounty program is an exciting opportunity for contributors to help us build the future of Zeta. By participating, you can earn rewards while contributing to a project that aims to revolutionize digital activity.
 4 | 
 5 | Here's how it works:
 6 | 
 7 | 1. **Check out our Roadmap**: We've shared our roadmap detailing our short and long-term goals. These are the areas where we're seeking contributions.
 8 | 
 9 | 2. **Pick a Task**: Choose a task from the roadmap that aligns with your skills and interests. If you're unsure, you can reach out to our team for guidance.
10 | 
11 | 3. **Get to Work**: Once you've chosen a task, start working on it. Remember, quality is key. We're looking for contributions that truly make a difference.
12 | 
13 | 4. **Submit your Contribution**: Once your work is complete, submit it for review. We'll evaluate your contribution based on its quality, relevance, and the value it brings to Zeta.
14 | 
15 | 5. **Earn Rewards**: If your contribution is approved, you'll earn a bounty. The amount of the bounty depends on the complexity of the task, the quality of your work, and the value it brings to Zeta.
16 | 
17 | ## The Three Phases of Our Bounty Program
18 | 
19 | ### Phase 1: Building the Foundation
20 | In the first phase, our focus is on building the basic infrastructure of Zeta. This includes developing key components like the Zeta class, integrating essential tools, and establishing task completion and evaluation logic. We'll also start developing our testing and evaluation framework during this phase. If you're interested in foundational work and have a knack for building robust, scalable systems, this phase is for you.
21 | 
22 | ### Phase 2: Enhancing the System
23 | In the second phase, we'll focus on enhancing Zeta by integrating more advanced features, improving the system's efficiency, and refining our testing and evaluation framework. This phase involves more complex tasks, so if you enjoy tackling challenging problems and contributing to the development of innovative features, this is the phase for you.
24 | 
25 | ### Phase 3: Towards Super-Intelligence
26 | The third phase of our bounty program is the most exciting - this is where we aim to achieve super-intelligence. In this phase, we'll be working on improving the swarm's capabilities, expanding its skills, and fine-tuning the system based on real-world testing and feedback. If you're excited about the future of AI and want to contribute to a project that could potentially transform the digital world, this is the phase for you.
27 | 
28 | Remember, our roadmap is a guide, and we encourage you to bring your own ideas and creativity to the table. We believe that every contribution, no matter how small, can make a difference. So join us on this exciting journey and help us create the future of Zeta.
29 | 
30 | **To participate in our bounty program, visit the [Zeta Bounty Program Page](https://ft.apac.ai/bounty).** Let's build the future together!
31 | 
32 | 
33 | 
34 | 
35 | 
36 | ## Bounties for Roadmap Items
37 | 
38 | To accelerate the development of Zeta and to encourage more contributors to join our journey towards automating every digital activity in existence, we are announcing a Bounty Program for specific roadmap items. Each bounty will be rewarded based on the complexity and importance of the task. Below are the items available for bounty:
39 | 
40 | 1. **Multi-Agent Debate Integration**: $2000
41 | 2. **Meta Prompting Integration**: $1500
42 | 3. **Zeta Class**: $1500
43 | 4. **Integration of Additional Tools**: $1000
44 | 5. **Task Completion and Evaluation Logic**: $2000
45 | 6. **Ocean Integration**: $2500
46 | 7. **Improved Communication**: $2000
47 | 8. **Testing and Evaluation**: $1500
48 | 9. **Worker Swarm Class**: $2000
49 | 10. **Documentation**: $500
50 | 
51 | For each bounty task, there will be a strict evaluation process to ensure the quality of the contribution. This process includes a thorough review of the code and extensive testing to ensure it meets our standards.
52 | 
53 | # 3-Phase Testing Framework
54 | 
55 | To ensure the quality and efficiency of the Swarm, we will introduce a 3-phase testing framework which will also serve as our evaluation criteria for each of the bounty tasks.
56 | 
57 | ## Phase 1: Unit Testing
58 | In this phase, individual modules will be tested to ensure that they work correctly in isolation. Unit tests will be designed for all functions and methods, with an emphasis on edge cases.
59 | 
60 | ## Phase 2: Integration Testing
61 | After passing unit tests, we will test the integration of different modules to ensure they work correctly together. This phase will also test the interoperability of the Swarm with external systems and libraries.
62 | 
63 | ## Phase 3: Benchmarking & Stress Testing
64 | In the final phase, we will perform benchmarking and stress tests. We'll push the limits of the Swarm under extreme conditions to ensure it performs well in real-world scenarios. This phase will measure the performance, speed, and scalability of the Swarm under high load conditions.
65 | 
66 | By following this 3-phase testing framework, we aim to develop a reliable, high-performing, and scalable Swarm that can automate all digital activities. 
67 | 
68 | # Reverse Engineering to Reach Phase 3
69 | 
70 | To reach the Phase 3 level, we need to reverse engineer the tasks we need to complete. Here's an example of what this might look like:
71 | 
72 | 1. **Set Clear Expectations**: Define what success looks like for each task. Be clear about the outputs and outcomes we expect. This will guide our testing and development efforts.
73 | 
74 | 2. **Develop Testing Scenarios**: Create a comprehensive list of testing scenarios that cover both common and edge cases. This will help us ensure that our Swarm can handle a wide range of situations.
75 | 
76 | 3. **Write Test Cases**: For each scenario, write detailed test cases that outline the exact steps to be followed, the inputs to be used, and the expected outputs.
77 | 
78 | 4. **Execute the Tests**: Run the test cases on our Swarm, making note of any issues or bugs that arise.
79 | 
80 | 5. **Iterate and Improve**: Based on the results of our tests, iterate and improve our Swarm. This may involve fixing bugs, optimizing code, or redesigning parts of our system.
81 | 
82 | 6. **Repeat**: Repeat this process until our Swarm meets our expectations and passes all test cases.
83 | 
84 | By following these steps, we will systematically build, test, and improve our Swarm until it reaches the Phase 3 level. This methodical approach will help us ensure that we create a reliable, high-performing, and scalable Swarm that can truly automate all digital activities.
85 | 
86 | Let's shape the future of digital automation together!
87 | 


--------------------------------------------------------------------------------
/docs/roadmap.md:
--------------------------------------------------------------------------------
  1 | 
  2 | **[Zeta's 3-Step Master Plan for Perfecting Multi-Modality LLMs]**
  3 | 
  4 | ---
  5 | 
  6 | **1. Refinement and Excellence: Perfecting the Framework**
  7 |     - **[Objective]**: To develop Zeta into the most sophisticated, yet intuitively simple framework for building Multi-Modality LLMs.
  8 | 
  9 |     - **[Strategies]**
 10 |         - **Zeta Innovation Labs**: 
 11 |             * Create a dedicated team of experts who exclusively focus on refining the foundational modules and blocks.
 12 |             * Prioritize research in areas like advanced self-supervised learning, multi-modal integration, and zero-shot learning.
 13 |         - **Modularity Focus**:
 14 |             * Develop plug-and-play modules that allow developers to effortlessly incorporate various data types (text, image, video, audio) into their LLMs.
 15 |             * Standardize the blocks ensuring consistent performance, error-handling, and interoperability.
 16 |         - **Performance Optimization**:
 17 |             * Collaborate with hardware manufacturers to ensure that Zeta is perfectly optimized for cutting-edge GPUs, TPUs, and other specialized hardware. 
 18 |             * Roll out regular updates to keep the framework at the forefront of performance.
 19 | 
 20 | ---
 21 | 
 22 | **2. User-Centric Development: Making Zeta Intuitive**
 23 |     - **[Objective]**: Ensure that every feature, tool, and module in Zeta aligns with the principle of making LLM creation simpler and more efficient.
 24 | 
 25 |     - **[Strategies]**
 26 |         - **Zeta Academy**:
 27 |             * Host frequent workshops and webinars targeted at educating users on harnessing the power of Zeta's multi-modality LLM features.
 28 |             * Create a vast library of tutorials, ranging from beginner to advanced, with real-world examples of LLM implementation.
 29 |         - **Interactive GUI for LLM Design**:
 30 |             * Develop a visual interface where users can drag-and-drop modules, visualize their LLM architecture, and see real-time performance metrics.
 31 |         - **Feedback Loops**:
 32 |             * Create a robust system to collect and implement feedback. Users should feel like they’re co-creating Zeta.
 33 |             * Launch a beta program where selected developers can test new features and provide insights.
 34 | 
 35 | ---
 36 | 
 37 | **3. Scaling and Outreach: From the Labs to the World**
 38 |     - **[Objective]**: Make Zeta the de facto choice for developers worldwide aiming to craft state-of-the-art Multi-Modality LLMs.
 39 | 
 40 |     - **[Strategies]**
 41 |         - **Zeta Ambassadors**:
 42 |             * Identify and collaborate with top AI researchers and practitioners globally, making them the face and voice of Zeta in their communities.
 43 |         - **Strategic Partnerships**:
 44 |             * Work closely with major tech institutions, universities, and platforms to integrate Zeta into their curriculum or platforms.
 45 |             * Create an API gateway for seamless integration of Zeta with other popular machine learning and data processing platforms.
 46 |         - **Global Challenges & Competitions**:
 47 |             * Organize worldwide LLM challenges, where developers use Zeta to solve real-world problems, bringing attention to both the problems and the capabilities of Zeta.
 48 | 
 49 | ---
 50 | 
 51 | 
 52 | In every tool, in every line of code, in every module of Zeta, you'll find our relentless pursuit of excellence. But remember, at its core, 
 53 | 
 54 | Zeta isn't about us,
 55 | 
 56 | it's about you, the creator. 
 57 | 
 58 | It's about giving you the power, the simplicity, and the edge to redefine the boundaries of what's possible. 
 59 | 
 60 | With Zeta, we’re not just building a tool; we're crafting the future. 
 61 | 
 62 | A future we're eager to see through your eyes.
 63 | 
 64 | 
 65 | 
 66 | 
 67 | ------
 68 | 
 69 | 
 70 | 
 71 | 
 72 | 
 73 | 
 74 | 
 75 | 
 76 | 
 77 | 
 78 | 
 79 | 
 80 | 
 81 | 
 82 | 
 83 | 
 84 | 
 85 | 
 86 | 
 87 | 
 88 | 
 89 | 
 90 | 
 91 | **[Zeta's 3-Step Master Plan]**
 92 | 
 93 | **1. Cultivate an Ecosystem of Innovation**
 94 |     - **[Objective]**: Establish an environment where creativity and innovation are paramount.
 95 |     
 96 |     - **[Strategies]**
 97 |         - **Education & Outreach**: 
 98 |             * Launch a series of free online courses, workshops, and webinars to educate developers on the capabilities and advantages of Zeta.
 99 |             * Partner with top universities and institutions, offering them early access and integrations, fostering a new generation of developers natively trained on Zeta.
100 |         - **Zeta Labs**: 
101 |             * Open a research lab committed to pushing the boundaries of what neural networks can achieve.
102 |             * Provide grants, resources, and mentorship to promising projects and startups that choose to build with Zeta.
103 |         - **Open Source Philosophy**:
104 |             * Release parts of Zeta's core codebase to the public, inviting developers worldwide to contribute, refine, and expand upon the framework.
105 |             * Organize hackathons and coding challenges to galvanize the community around real-world problems that Zeta can solve.
106 | 
107 | ---
108 | 
109 | **2. Seamless Integration & Scalability**
110 |     - **[Objective]**: Make Zeta the easiest, most efficient, and most scalable framework to integrate into any project or system.
111 |     
112 |     - **[Strategies]**
113 |         - **Developer Toolkits**:
114 |             * Release a suite of tools, plugins, and libraries for all major development platforms and languages, ensuring Zeta is accessible to everyone, everywhere.
115 |         - **Zeta Cloud**:
116 |             * Offer a cloud solution that allows developers to run, test, and deploy their neural networks seamlessly. This ensures businesses of all sizes can scale without friction.
117 |         - **Partnerships**:
118 |             * Collaborate with major tech companies, ensuring Zeta's native support on platforms like AWS, Google Cloud, and Azure.
119 |             * Establish alliances with hardware manufacturers, optimizing Zeta for the latest GPUs and Neural Network Processors.
120 | 
121 | ---
122 | 
123 | **3. Build a Community and Cultivate Trust**
124 |     - **[Objective]**: Establish Zeta as more than a tool – it should be a movement, a community of forward-thinkers who believe in redefining the boundaries of neural network capabilities.
125 |     
126 |     - **[Strategies]**
127 |         - **ZetaCon**:
128 |             * Annually host a global conference (both offline and online) bringing together the brightest minds in the AI and machine learning sector. It will be a platform for networking, knowledge-sharing, and showcasing the best of what's been built using Zeta.
129 |         - **Transparency Reports**:
130 |             * Release regular updates about Zeta's development, challenges, successes, and roadmap.
131 |             * Actively gather feedback, ensuring the community feels heard and that their insights are valued.
132 |         - **Zeta Academy**:
133 |             * Create a platform where developers can share their projects, tutorials, and courses about Zeta. Recognize and reward the best contributions to foster a sense of ownership and pride within the community.
134 | 
135 | ---
136 | 
137 | This isn't just a roadmap. It's our promise, our commitment. Because at the end of the day, it's not about the lines of code we write. It's about the lives we change, the innovations we inspire, and the future we create. And with Zeta, we believe that future is brighter than ever. Let's build it together.
138 | 
139 | 
140 | 


--------------------------------------------------------------------------------
/docs/flywheel.md:
--------------------------------------------------------------------------------
  1 | # The Zeta Flywheel
  2 | 
  3 | 1. **Building a Supportive Community:** Initiate by establishing an engaging and inclusive open-source community for both developers and sales freelancers around Zeta. Regular online meetups, webinars, tutorials, and sales training can make them feel welcome and encourage contributions and sales efforts.
  4 | 
  5 | 2. **Increased Contributions and Sales Efforts:** The more engaged the community, the more developers will contribute to Zeta and the more effort sales freelancers will put into selling Zeta.
  6 | 
  7 | 3. **Improvement in Quality and Market Reach:** More developer contributions mean better quality, reliability, and feature offerings from Zeta. Simultaneously, increased sales efforts from freelancers boost Zeta' market penetration and visibility.
  8 | 
  9 | 4. **Rise in User Base:** As Zeta becomes more robust and more well-known, the user base grows, driving more revenue.
 10 | 
 11 | 5. **Greater Financial Incentives:** Increased revenue can be redirected to offer more significant financial incentives to both developers and salespeople. Developers can be incentivized based on their contribution to Zeta, and salespeople can be rewarded with higher commissions.
 12 | 
 13 | 6. **Attract More Developers and Salespeople:** These financial incentives, coupled with the recognition and experience from participating in a successful project, attract more developers and salespeople to the community.
 14 | 
 15 | 7. **Wider Adoption of Zeta:** An ever-improving product, a growing user base, and an increasing number of passionate salespeople accelerate the adoption of Zeta.
 16 | 
 17 | 8. **Return to Step 1:** As the community, user base, and sales network continue to grow, the cycle repeats, each time speeding up the flywheel.
 18 | 
 19 | 
 20 | ```markdown
 21 |                +---------------------+
 22 |                |   Building a       |
 23 |                |  Supportive        | <--+
 24 |                |   Community        |    |
 25 |                +--------+-----------+    |
 26 |                         |                |
 27 |                         v                |
 28 |                +--------+-----------+    |
 29 |                |   Increased        |    |
 30 |                | Contributions &    |    |
 31 |                |   Sales Efforts    |    |
 32 |                +--------+-----------+    |
 33 |                         |                |
 34 |                         v                |
 35 |                +--------+-----------+    |
 36 |                |   Improvement in   |    |
 37 |                | Quality & Market   |    |
 38 |                |       Reach        |    |
 39 |                +--------+-----------+    |
 40 |                         |                |
 41 |                         v                |
 42 |                +--------+-----------+    |
 43 |                |   Rise in User     |    |
 44 |                |        Base        |    |
 45 |                +--------+-----------+    |
 46 |                         |                |
 47 |                         v                |
 48 |                +--------+-----------+    |
 49 |                |  Greater Financial |    |
 50 |                |     Incentives     |    |
 51 |                +--------+-----------+    |
 52 |                         |                |
 53 |                         v                |
 54 |                +--------+-----------+    |
 55 |                | Attract More        |    |
 56 |                | Developers &       |    |
 57 |                | Salespeople         |    |
 58 |                +--------+-----------+    |
 59 |                         |                |
 60 |                         v                |
 61 |                +--------+-----------+    |
 62 |                |  Wider Adoption of  |    |
 63 |                |       Zeta        |----+
 64 |                +---------------------+
 65 | ```
 66 | 
 67 | 
 68 | # Potential Risks and Mitigations:
 69 | 
 70 | 1. **Insufficient Contributions or Quality of Work**: Open-source efforts rely on individuals being willing and able to spend time contributing. If not enough people participate, or the work they produce is of poor quality, the product development could stall. 
 71 |    * **Mitigation**: Create a robust community with clear guidelines, support, and resources. Provide incentives for quality contributions, such as a reputation system, swag, or financial rewards. Conduct thorough code reviews to ensure the quality of contributions.
 72 | 
 73 | 2. **Lack of Sales Results**: Commission-based salespeople will only continue to sell the product if they're successful. If they aren't making enough sales, they may lose motivation and cease their efforts.
 74 |    * **Mitigation**: Provide adequate sales training and resources. Ensure the product-market fit is strong, and adjust messaging or sales tactics as necessary. Consider implementing a minimum commission or base pay to reduce risk for salespeople.
 75 | 
 76 | 3. **Poor User Experience or User Adoption**: If users don't find the product useful or easy to use, they won't adopt it, and the user base won't grow. This could also discourage salespeople and contributors.
 77 |    * **Mitigation**: Prioritize user experience in the product development process. Regularly gather and incorporate user feedback. Ensure robust user support is in place.
 78 | 
 79 | 4. **Inadequate Financial Incentives**: If the financial rewards don't justify the time and effort contributors and salespeople are putting in, they will likely disengage.
 80 |    * **Mitigation**: Regularly review and adjust financial incentives as needed. Ensure that the method for calculating and distributing rewards is transparent and fair.
 81 | 
 82 | 5. **Security and Compliance Risks**: As the user base grows and the software becomes more complex, the risk of security issues increases. Moreover, as contributors from various regions join, compliance with various international laws could become an issue.
 83 |    * **Mitigation**: Establish strong security practices from the start. Regularly conduct security audits. Seek legal counsel to understand and adhere to international laws and regulations.
 84 | 
 85 | ## Activation Plan for the Flywheel:
 86 | 
 87 | 1. **Community Building**: Begin by fostering a supportive community around Zeta. Encourage early adopters to contribute and provide feedback. Create comprehensive documentation, community guidelines, and a forum for discussion and support.
 88 | 
 89 | 2. **Sales and Development Training**: Provide resources and training for salespeople and developers. Make sure they understand the product, its value, and how to effectively contribute or sell.
 90 | 
 91 | 3. **Increase Contributions and Sales Efforts**: Encourage increased participation by highlighting successful contributions and sales, rewarding top contributors and salespeople, and regularly communicating about the project's progress and impact.
 92 | 
 93 | 4. **Iterate and Improve**: Continually gather and implement feedback to improve Zeta and its market reach. The better the product and its alignment with the market, the more the user base will grow.
 94 | 
 95 | 5. **Expand User Base**: As the product improves and sales efforts continue, the user base should grow. Ensure you have the infrastructure to support this growth and maintain a positive user experience.
 96 | 
 97 | 6. **Increase Financial Incentives**: As the user base and product grow, so too should the financial incentives. Make sure rewards continue to be competitive and attractive.
 98 | 
 99 | 7. **Attract More Contributors and Salespeople**: As the financial incentives and success of the product increase, this should attract more contributors and salespeople, further feeding the flywheel.
100 | 
101 | Throughout this process, it's important to regularly reassess and adjust your strategy as necessary. Stay flexible and responsive to changes in the market, user feedback, and the evolving needs of the community.


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
  1 |                                  Apache License
  2 |                            Version 2.0, January 2004
  3 |                         http://www.apache.org/licenses/
  4 | 
  5 |    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
  6 | 
  7 |    1. Definitions.
  8 | 
  9 |       "License" shall mean the terms and conditions for use, reproduction,
 10 |       and distribution as defined by Sections 1 through 9 of this document.
 11 | 
 12 |       "Licensor" shall mean the copyright owner or entity authorized by
 13 |       the copyright owner that is granting the License.
 14 | 
 15 |       "Legal Entity" shall mean the union of the acting entity and all
 16 |       other entities that control, are controlled by, or are under common
 17 |       control with that entity. For the purposes of this definition,
 18 |       "control" means (i) the power, direct or indirect, to cause the
 19 |       direction or management of such entity, whether by contract or
 20 |       otherwise, or (ii) ownership of fifty percent (50%) or more of the
 21 |       outstanding shares, or (iii) beneficial ownership of such entity.
 22 | 
 23 |       "You" (or "Your") shall mean an individual or Legal Entity
 24 |       exercising permissions granted by this License.
 25 | 
 26 |       "Source" form shall mean the preferred form for making modifications,
 27 |       including but not limited to software source code, documentation
 28 |       source, and configuration files.
 29 | 
 30 |       "Object" form shall mean any form resulting from mechanical
 31 |       transformation or translation of a Source form, including but
 32 |       not limited to compiled object code, generated documentation,
 33 |       and conversions to other media types.
 34 | 
 35 |       "Work" shall mean the work of authorship, whether in Source or
 36 |       Object form, made available under the License, as indicated by a
 37 |       copyright notice that is included in or attached to the work
 38 |       (an example is provided in the Appendix below).
 39 | 
 40 |       "Derivative Works" shall mean any work, whether in Source or Object
 41 |       form, that is based on (or derived from) the Work and for which the
 42 |       editorial revisions, annotations, elaborations, or other modifications
 43 |       represent, as a whole, an original work of authorship. For the purposes
 44 |       of this License, Derivative Works shall not include works that remain
 45 |       separable from, or merely link (or bind by name) to the interfaces of,
 46 |       the Work and Derivative Works thereof.
 47 | 
 48 |       "Contribution" shall mean any work of authorship, including
 49 |       the original version of the Work and any modifications or additions
 50 |       to that Work or Derivative Works thereof, that is intentionally
 51 |       submitted to Licensor for inclusion in the Work by the copyright owner
 52 |       or by an individual or Legal Entity authorized to submit on behalf of
 53 |       the copyright owner. For the purposes of this definition, "submitted"
 54 |       means any form of electronic, verbal, or written communication sent
 55 |       to the Licensor or its representatives, including but not limited to
 56 |       communication on electronic mailing lists, source code control systems,
 57 |       and issue tracking systems that are managed by, or on behalf of, the
 58 |       Licensor for the purpose of discussing and improving the Work, but
 59 |       excluding communication that is conspicuously marked or otherwise
 60 |       designated in writing by the copyright owner as "Not a Contribution."
 61 | 
 62 |       "Contributor" shall mean Licensor and any individual or Legal Entity
 63 |       on behalf of whom a Contribution has been received by Licensor and
 64 |       subsequently incorporated within the Work.
 65 | 
 66 |    2. Grant of Copyright License. Subject to the terms and conditions of
 67 |       this License, each Contributor hereby grants to You a perpetual,
 68 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 69 |       copyright license to reproduce, prepare Derivative Works of,
 70 |       publicly display, publicly perform, sublicense, and distribute the
 71 |       Work and such Derivative Works in Source or Object form.
 72 | 
 73 |    3. Grant of Patent License. Subject to the terms and conditions of
 74 |       this License, each Contributor hereby grants to You a perpetual,
 75 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 76 |       (except as stated in this section) patent license to make, have made,
 77 |       use, offer to sell, sell, import, and otherwise transfer the Work,
 78 |       where such license applies only to those patent claims licensable
 79 |       by such Contributor that are necessarily infringed by their
 80 |       Contribution(s) alone or by combination of their Contribution(s)
 81 |       with the Work to which such Contribution(s) was submitted. If You
 82 |       institute patent litigation against any entity (including a
 83 |       cross-claim or counterclaim in a lawsuit) alleging that the Work
 84 |       or a Contribution incorporated within the Work constitutes direct
 85 |       or contributory patent infringement, then any patent licenses
 86 |       granted to You under this License for that Work shall terminate
 87 |       as of the date such litigation is filed.
 88 | 
 89 |    4. Redistribution. You may reproduce and distribute copies of the
 90 |       Work or Derivative Works thereof in any medium, with or without
 91 |       modifications, and in Source or Object form, provided that You
 92 |       meet the following conditions:
 93 | 
 94 |       (a) You must give any other recipients of the Work or
 95 |           Derivative Works a copy of this License; and
 96 | 
 97 |       (b) You must cause any modified files to carry prominent notices
 98 |           stating that You changed the files; and
 99 | 
100 |       (c) You must retain, in the Source form of any Derivative Works
101 |           that You distribute, all copyright, patent, trademark, and
102 |           attribution notices from the Source form of the Work,
103 |           excluding those notices that do not pertain to any part of
104 |           the Derivative Works; and
105 | 
106 |       (d) If the Work includes a "NOTICE" text file as part of its
107 |           distribution, then any Derivative Works that You distribute must
108 |           include a readable copy of the attribution notices contained
109 |           within such NOTICE file, excluding those notices that do not
110 |           pertain to any part of the Derivative Works, in at least one
111 |           of the following places: within a NOTICE text file distributed
112 |           as part of the Derivative Works; within the Source form or
113 |           documentation, if provided along with the Derivative Works; or,
114 |           within a display generated by the Derivative Works, if and
115 |           wherever such third-party notices normally appear. The contents
116 |           of the NOTICE file are for informational purposes only and
117 |           do not modify the License. You may add Your own attribution
118 |           notices within Derivative Works that You distribute, alongside
119 |           or as an addendum to the NOTICE text from the Work, provided
120 |           that such additional attribution notices cannot be construed
121 |           as modifying the License.
122 | 
123 |       You may add Your own copyright statement to Your modifications and
124 |       may provide additional or different license terms and conditions
125 |       for use, reproduction, or distribution of Your modifications, or
126 |       for any such Derivative Works as a whole, provided Your use,
127 |       reproduction, and distribution of the Work otherwise complies with
128 |       the conditions stated in this License.
129 | 
130 |    5. Submission of Contributions. Unless You explicitly state otherwise,
131 |       any Contribution intentionally submitted for inclusion in the Work
132 |       by You to the Licensor shall be under the terms and conditions of
133 |       this License, without any additional terms or conditions.
134 |       Notwithstanding the above, nothing herein shall supersede or modify
135 |       the terms of any separate license agreement you may have executed
136 |       with Licensor regarding such Contributions.
137 | 
138 |    6. Trademarks. This License does not grant permission to use the trade
139 |       names, trademarks, service marks, or product names of the Licensor,
140 |       except as required for reasonable and customary use in describing the
141 |       origin of the Work and reproducing the content of the NOTICE file.
142 | 
143 |    7. Disclaimer of Warranty. Unless required by applicable law or
144 |       agreed to in writing, Licensor provides the Work (and each
145 |       Contributor provides its Contributions) on an "AS IS" BASIS,
146 |       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 |       implied, including, without limitation, any warranties or conditions
148 |       of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 |       PARTICULAR PURPOSE. You are solely responsible for determining the
150 |       appropriateness of using or redistributing the Work and assume any
151 |       risks associated with Your exercise of permissions under this License.
152 | 
153 |    8. Limitation of Liability. In no event and under no legal theory,
154 |       whether in tort (including negligence), contract, or otherwise,
155 |       unless required by applicable law (such as deliberate and grossly
156 |       negligent acts) or agreed to in writing, shall any Contributor be
157 |       liable to You for damages, including any direct, indirect, special,
158 |       incidental, or consequential damages of any character arising as a
159 |       result of this License or out of the use or inability to use the
160 |       Work (including but not limited to damages for loss of goodwill,
161 |       work stoppage, computer failure or malfunction, or any and all
162 |       other commercial damages or losses), even if such Contributor
163 |       has been advised of the possibility of such damages.
164 | 
165 |    9. Accepting Warranty or Additional Liability. While redistributing
166 |       the Work or Derivative Works thereof, You may choose to offer,
167 |       and charge a fee for, acceptance of support, warranty, indemnity,
168 |       or other liability obligations and/or rights consistent with this
169 |       License. However, in accepting such obligations, You may act only
170 |       on Your own behalf and on Your sole responsibility, not on behalf
171 |       of any other Contributor, and only if You agree to indemnify,
172 |       defend, and hold each Contributor harmless for any liability
173 |       incurred by, or claims asserted against, such Contributor by reason
174 |       of your accepting any such warranty or additional liability.
175 | 
176 |    END OF TERMS AND CONDITIONS
177 | 
178 |    APPENDIX: How to apply the Apache License to your work.
179 | 
180 |       To apply the Apache License to your work, attach the following
181 |       boilerplate notice, with the fields enclosed by brackets "[]"
182 |       replaced with your own identifying information. (Don't include
183 |       the brackets!)  The text should be enclosed in the appropriate
184 |       comment syntax for the file format. We also recommend that a
185 |       file or class name and description of purpose be included on the
186 |       same "printed page" as the copyright notice for easier
187 |       identification within third-party archives.
188 | 
189 |    Copyright [yyyy] [name of copyright owner]
190 | 
191 |    Licensed under the Apache License, Version 2.0 (the "License");
192 |    you may not use this file except in compliance with the License.
193 |    You may obtain a copy of the License at
194 | 
195 |        http://www.apache.org/licenses/LICENSE-2.0
196 | 
197 |    Unless required by applicable law or agreed to in writing, software
198 |    distributed under the License is distributed on an "AS IS" BASIS,
199 |    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200 |    See the License for the specific language governing permissions and
201 |    limitations under the License.
202 | 


--------------------------------------------------------------------------------
/docs/ft/finetuner.md:
--------------------------------------------------------------------------------
  1 | # FineTuner Class
  2 | 
  3 | ## Overview and Introduction
  4 | The FineTuner class is a part of the finetuning suite, designed for fine-tuning pre-trained language models for various natural language processing tasks. This class serves as a wrapper for several functionalities, including data preprocessing, model training, and text generation using a pre-trained model. The main components include the HuggingFace's transformers library, the peft library for task types, and the datasets library for loading datasets. 
  5 | 
  6 | The FineTuner class is designed to streamline the process of fine-tuning by handling various configurations and components such as LoRA (Low-Rank Adaptation), quantization, and inference. This is especially important in the context of large-scale models and datasets where fine-tuning needs to be efficient and adaptable.
  7 | 
  8 | ## Class Definition
  9 | 
 10 | ```python
 11 | class FineTuner:
 12 |     def __init__(self, 
 13 |                 model_id: str, 
 14 |                 device: str = None,
 15 |                 dataset_name=None, 
 16 |                 lora_r=16,
 17 |                 lora_alpha=32,
 18 |                 lora_target_modules=["q", "v"],
 19 |                 lora_bias="none",
 20 |                 preprocessor=None,
 21 |                 lora_task_type=TaskType.SEQ_2_SEQ_LM,
 22 |                 max_length=1000, 
 23 |                 quantize: bool = False, 
 24 |                 quantization_config: dict = None,
 25 |                 trainer_config=None,
 26 |                 inference_handler=None
 27 |             ):
 28 |     ...
 29 | 
 30 | ```
 31 | 
 32 | ### Parameters:
 33 | 
 34 | - `model_id` (str): The identifier of the pre-trained model to be fine-tuned.
 35 | - `device` (str, optional): The device to run the model on, either 'cpu' or 'cuda'. Default is 'cuda' if available, otherwise 'cpu'.
 36 | - `dataset_name` (str, optional): The name of the dataset to be used for training. Default is `None`.
 37 | - `lora_r` (int, optional): The rank of LoRA layers. Default is 16.
 38 | - `lora_alpha` (int, optional): The over-parameterization ratio of LoRA layers. Default is 32.
 39 | - `lora_target_modules` (list, optional): The target modules for LoRA. Default is ["q", "v"].
 40 | - `lora_bias` (str, optional): The bias of LoRA. Default is "none".
 41 | - `preprocessor` (Preprocessor, optional): The preprocessor for tokenizing the dataset. Default is `DefaultPreprocessor`.
 42 | - `lora_task_type` (TaskType, optional): The task type for LoRA. Default is `TaskType.SEQ_2_SEQ_LM`.
 43 | - `max_length` (int, optional): The maximum length of the generated text. Default is 1000.
 44 | - `quantize` (bool, optional): Whether to quantize the model weights. Default is `False`.
 45 | - `quantization_config` (dict, optional): The configuration for quantization. Default is `None`.
 46 | - `trainer_config` (TrainerConfig, optional): The configuration for the trainer. Default is `DefaultTrainerConfig`.
 47 | - `inference_handler` (InferenceHandler, optional): The handler for inference. Default is `DefaultInferenceHandler`.
 48 | 
 49 | ### Methods:
 50 | 
 51 | #### `__call__(self, prompt_text: str, max_length: int = None) -> str`
 52 | 
 53 | Generates text based on the provided `prompt_text`.
 54 | 
 55 | Parameters:
 56 | - `prompt_text` (str): The text prompt to base the generation on.
 57 | - `max_length` (int, optional): The maximum length of the generated text. Default is `self.max_length`.
 58 | 
 59 | Returns:
 60 | - `str`: The generated text.
 61 | 
 62 | #### `preprocess_data(self)`
 63 | 
 64 | Preprocesses the dataset by tokenizing the text and removing unnecessary columns.
 65 | 
 66 | Returns:
 67 | - `Dataset`: The tokenized dataset.
 68 | 
 69 | #### `train(self, output_dir, num_train_epochs)`
 70 | 
 71 | Trains the model on the preprocessed dataset.
 72 | 
 73 | Parameters:
 74 | - `output_dir` (str): The directory to save the trained model.
 75 | - `num_train_epochs` (int): The number of epochs to train the model.
 76 | 
 77 | #### `generate(self, prompt_text: str, max_length: int = None) -> str`
 78 | 
 79 | Generates text based on the provided `prompt_text` using the `inference_handler`.
 80 | 
 81 | Parameters:
 82 | - `prompt_text` (str): The text prompt to base the generation on.
 83 | - `max_length` (int, optional): The maximum length of the generated text. Default is `self.max_length`.
 84 | 
 85 | Returns:
 86 | - `str`: The generated text.
 87 | 
 88 | ## Functionality and Usage
 89 | 
 90 | ### Preprocessing Data
 91 | Before training the model, the data needs to be preprocessed. The `preprocess_data` method tokenizes the dataset using the specified preprocessor. It removes the unnecessary columns and returns the tokenized dataset.
 92 | 
 93 | ### Training the Model
 94 | The `train` method trains the model using the specified trainer configuration, preprocessed dataset, and training arguments. It configures the model, tokenizer, and training arguments using the `trainer_config` object, and then initializes the `Seq2SeqTrainer` with the configured model, training arguments, data collator, and training dataset. It then trains the model using the `train` method of the `Seq2SeqTrainer`.
 95 | 
 96 | ### Generating Text
 97 | The `generate` method generates text based on the provided prompt text using the specified inference handler. It calls the `generate` method of the `inference_handler` object with the specified prompt text, model, tokenizer, device, and maximum length.
 98 | 
 99 | 
100 | ## Additional Information and Tips
101 | 
102 | ### Quantization
103 | Quantization is the process of converting the weights and biases of the model from floating point to integer values. This is useful for reducing the memory requirements and speeding up the model inference. The `quantize` parameter specifies whether to quantize the model or not, and the `quantization_config` parameter specifies the configuration for quantization. The default quantization configuration is 4-bit quantization with double quantization and "nf4" quantization type.
104 | 
105 | ### Low-Rank Adaptation (LoRA)
106 | LoRA is a method for fine-tuning pre-trained models with a small number of additional parameters. The `lora_r` parameter specifies the rank for LoRA, the `lora_alpha` parameter specifies the scaling factor for LoRA, the `lora_target_modules` parameter specifies the target modules for LoRA, and the `lora_bias` parameter specifies the bias for LoRA.
107 | 
108 | 
109 | 
110 | ### Usage Examples:
111 | 
112 | #### Example 1:
113 | 
114 | ```python
115 | from finetuning_suite import FineTuner
116 | import torch
117 | 
118 | # Initialize the FineTuner
119 | finetuner = FineTuner(
120 |     model_id="gpt2",
121 |     device="cuda",
122 |     dataset_name="dialogue",
123 |     lora_r=16,
124 |     lora_alpha=32,
125 |     max_length=1000,
126 |     quantize=True
127 | )
128 | 
129 | # Preprocess the data
130 | tokenized_dataset = finetuner.preprocess_data()
131 | 
132 | # Train the model
133 | output_dir = "./trained_model"
134 | num_train_epochs = 3
135 | finetuner.train(output_dir, num_train_epochs)
136 | 
137 | # Generate text
138 | prompt_text = "Once upon a time"
139 | generated_text = finetuner.generate(prompt_text)
140 | print(generated_text)
141 | ```
142 | 
143 | #### Example 2:
144 | 
145 | ```python
146 | from finetuning_suite import FineTuner
147 | import torch
148 | 
149 | # Initialize the FineTuner with custom quantization configuration
150 | quantization_config = {
151 |     'load_in_4bit': True,
152 |     'bnb_4bit_use_double_quant': True,
153 |     'bnb_4bit_quant_type': "nf4",
154 |     'bnb_4bit_compute_dtype': torch.bfloat16
155 | }
156 | 
157 | finetuner = FineTuner(
158 |     model_id="gpt2",
159 |     device="cuda",
160 |     dataset_name="dialogue",
161 |     max_length=500,
162 |     quantize=True,
163 |     quantization_config=quantization_config
164 | )
165 | 
166 | # Generate text
167 | prompt_text = "Once upon a time"
168 | generated_text = finetuner.generate(prompt_text)
169 | print(generated_text)
170 | ```
171 | 
172 | #### Example 3:
173 | 
174 | ```python
175 | from finetuning_suite import FineTuner
176 | 
177 | # Initialize the FineTuner with custom trainer configuration
178 | trainer_config = DefaultTrainerConfig(
179 |     output_dir="./trained_model",
180 |     num_train_epochs=3,
181 |     per_device_train_batch_size=8,
182 |     save_steps=10_000,
183 |     save_total_limit=2,
184 | )
185 | 
186 | finetuner = FineTuner(
187 |     model_id="gpt2",
188 |     device="cuda",
189 |     dataset_name="dialogue",
190 |     max_length=1000,
191 |     trainer_config=trainer_config
192 | )
193 | 
194 | # Train the model
195 | finetuner.train()
196 | 
197 | # Generate text
198 | prompt_text = "Once upon a time"
199 | generated_text = finetuner.generate(prompt_text)
200 | print(generated_text)
201 | ```
202 | 
203 | ### Mathematical Formulation:
204 | 
205 | The `FineTuner` class includes several components, each with its own mathematical formulation:
206 | 
207 | 1. LoRA (Low-Rank Adaptation): LoRA is a technique to adapt pre-trained models for new tasks with limited data. Given a weight matrix \(W \in \mathbb{R}^{m \times n}\) of a pre-trained model, LoRA decomposes \(W\) into low-rank and diagonal components:
208 | 
209 | \[ W = UDV^T + \Delta \]
210 | 
211 | Where:
212 | - \( U \in \mathbb{R}^{m \times r} \) and \( V \in \mathbb{R}^{n \times r} \) are low-rank matrices.
213 | - \( D \in \mathbb{R}^{r \times r} \) is a diagonal matrix.
214 | - \( \Delta \in \mathbb{R}^{m \times n} \)
215 | 
216 |  is a residual matrix.
217 | 
218 | 2. Quantization: The `FineTuner` class supports quantizing the model weights to reduce the model size and accelerate inference. The quantization process involves mapping the full-precision weights to a smaller set of quantized values. For example, in 4-bit quantization, the weights are mapped to one of 16 possible values.
219 | 
220 | 3. Text Generation: The `FineTuner` class generates text using a causal language model. Given a prompt text, the model predicts the next word in the sequence until the maximum length is reached or a stop token is generated. The probability of each word in the vocabulary is computed using the softmax function:
221 | 
222 | \[ P(w_i | w_1, ..., w_{i-1}) = \frac{e^{z_i}}{\sum_{j=1}^{V} e^{z_j}} \]
223 | 
224 | Where:
225 | - \( w_i \) is the ith word in the sequence.
226 | - \( z_i \) is the logit for the ith word in the vocabulary.
227 | - \( V \) is the size of the vocabulary.
228 | 
229 | ### Implementation Details:
230 | 
231 | The `FineTuner` class is implemented using the Hugging Face `transformers` library. The `transformers` library provides pre-trained models, tokenizers, and training utilities for natural language processing tasks.
232 | 
233 | The `FineTuner` class includes the following components:
234 | 
235 | 1. Preprocessing: The `preprocess_data` method tokenizes the dataset using the tokenizer associated with the specified `model_id`. The `DefaultPreprocessor` class is used for tokenization by default, but a custom preprocessor can be specified using the `preprocessor` parameter.
236 | 
237 | 2. Training: The `train` method fine-tunes the model on the preprocessed dataset. The `DefaultTrainer` class from the `transformers` library is used for training by default, but a custom trainer can be specified using the `trainer_config` parameter.
238 | 
239 | 3. Inference: The `generate` method generates text based on a provided `prompt_text`. The `DefaultInferenceHandler` class is used for inference by default, but a custom inference handler can be specified using the `inference_handler` parameter.
240 | 
241 | 4. Quantization: The `FineTuner` class supports quantizing the model weights to reduce the model size and accelerate inference. The `quantize` parameter determines whether to quantize the model weights, and the `quantization_config` parameter specifies the configuration for quantization.
242 | 
243 | ### Limitations:
244 | 
245 | The `FineTuner` class has some limitations:
246 | 
247 | 1. Memory Consumption: Fine-tuning large models requires a significant amount of GPU memory. It is recommended to use a GPU with at least 16 GB of memory for fine-tuning large models.
248 | 
249 | 2. Computation Time: Fine-tuning large models requires a significant amount of computation time. It is recommended to use a powerful GPU to accelerate the training process.
250 | 
251 | 3. Quantization Accuracy: Quantizing the model weights reduces the model size and accelerates inference, but may also result in a slight decrease in model accuracy. It is recommended to evaluate the quantized model on a validation set to ensure that the accuracy is acceptable for the specific application.
252 | 
253 | 
254 | ------
255 | 
256 | ## Custom Finetuning, Inference, and Data preprocessing logic
257 | 
258 | #### `TrainerConfiguration` Abstract Class
259 | 
260 | The `TrainerConfiguration` abstract class is designed to offer flexibility to users, enabling them to define custom training configurations and strategies to be used with the `FineTuner` class.
261 | 
262 | - **configure(model, tokenizer, output_dir, num_train_epochs)**: This method is responsible for configuring the model, data collator, and training arguments.
263 | 
264 | #### Usage:
265 | 
266 | 1. **Extend TrainerConfiguration**:
267 |    
268 |    Create your custom trainer configuration by extending `TrainerConfiguration`:
269 | 
270 |    ```python
271 |    class MyTrainerConfig(TrainerConfiguration):
272 |        def configure(self, model, tokenizer, output_dir, num_train_epochs):
273 |            ...
274 |            return model, data_collator, training_args
275 |    ```
276 | 
277 | 2. **Pass to FineTuner**:
278 |    
279 |    Once you've defined your configuration, pass it to the `FineTuner`:
280 | 
281 |    ```python
282 |    my_config = MyTrainerConfig()
283 |    finetuner = FineTuner(model_id='your_model_id', trainer_config=my_config)
284 |    ```
285 | 
286 | This setup makes the `FineTuner` extremely flexible, accommodating various training strategies and configurations as required by the user.
287 | 
288 | -----
289 | 
290 | ## Documentation
291 | 
292 | ### Overview
293 | 
294 | Our system is architected to be modular, making it versatile and customizable at various stages of the fine-tuning process. Three primary components encapsulate the critical steps: 
295 | 1. **Preprocessing**: Transform raw input data into a format suitable for training.
296 | 2. **Training Configuration**: Set the training parameters, model adjustments, and collators.
297 | 3. **Inference**: Generate outputs based on user-provided input.
298 | 
299 | The entire system relies on abstract base classes, allowing developers to create custom implementations for each of the aforementioned steps.
300 | 
301 | ### 1. Preprocessing 
302 | 
303 | #### Abstract Base Class: `Preprocessor`
304 | - **Initial Parameters**:
305 |   - `tokenizer`: The tokenizer associated with the model.
306 |   
307 | - **Methods**:
308 |   - `preprocess_function(sample, padding="max_length")`: Transforms the input data sample to a format suitable for model input.
309 | 
310 | #### Default Implementation: `DefaultPreprocessor`
311 | Converts dialogues into summaries, tokenizes them, and manages padding/truncation.
312 | 
313 | #### Customization
314 | To create a custom preprocessor, inherit from the `Preprocessor` class and implement the `preprocess_function` method.
315 | 
316 | ### 2. Training Configuration
317 | 
318 | #### Abstract Base Class: `TrainerConfiguration`
319 | - **Methods**:
320 |   - `configure(model, tokenizer, output_dir, num_train_epochs, *args, **kwargs)`: Configures the model, data collator, and training arguments.
321 | 
322 | #### Default Implementation: `DefaultTrainerConfig`
323 | Uses LoRA configurations and sets up a `Seq2Seq` collator and training arguments.
324 | 
325 | #### Customization
326 | Inherit from `TrainerConfiguration` and implement the `configure` method to customize the training setup.
327 | 
328 | ### 3. Inference 
329 | 
330 | #### Abstract Base Class: `InferenceHandler`
331 | - **Methods**:
332 |   - `generate(prompt_text, model, tokenizer, device, max_length)`: Processes the prompt text and uses the model to generate an output.
333 | 
334 | #### Default Implementation: `DefaultInferenceHandler`
335 | Encodes the prompt, generates sequences with the model, and decodes the output.
336 | 
337 | #### Customization
338 | Developers can inherit from `InferenceHandler` and implement the `generate` method to customize the inference logic.
339 | 
340 | ---
341 | 
342 | ### Examples
343 | 
344 | 1. **Custom Preprocessor**:
345 | ```python
346 | class MyPreprocessor(Preprocessor):
347 |     def preprocess_function(self, sample, padding="max_length"):
348 |         # Custom preprocessing logic here
349 |         ...
350 |         return processed_sample
351 | ```
352 | 
353 | 2. **Custom Trainer Configuration**:
354 | ```python
355 | class MyTrainerConfig(TrainerConfiguration):
356 |     def configure(self, model, tokenizer, output_dir, num_train_epochs, *args, **kwargs):
357 |         # Custom training configuration logic here
358 |         ...
359 |         return custom_model, custom_data_collator, custom_training_args
360 | ```
361 | 
362 | 3. **Custom Inference Handler**:
363 | ```python
364 | class MyInferenceHandler(InferenceHandler):
365 |     def generate(self, prompt_text, model, tokenizer, device, max_length):
366 |         # Custom inference logic here
367 |         ...
368 |         return custom_output
369 | ```
370 | 
371 | ### Conclusion
372 | This documentation provides a roadmap for creating custom implementations for preprocessing, training, and inference logic. The modular architecture ensures flexibility and promotes adherence to the open/closed principle, making the system easily extensible without modifying existing code. Ensure your custom classes inherit from the appropriate base class and implement the required methods for seamless integration.
373 | 
374 | 
375 | 
376 | 
377 | 
378 | 
379 | -------
380 | 
381 | 
382 | 
383 | # Custom Preprocesing aDocumentation
384 | 
385 | ### `Preprocessor` Abstract Class Documentation
386 | 
387 | #### Overview:
388 | The `Preprocessor` abstract class serves as a blueprint for custom data preprocessing strategies to be used with the `FineTuner` class. The primary goal is to provide a polymorphic structure that enables users to create their custom preprocessing functions while adhering to the established interface.
389 | 
390 | #### Structure:
391 | - The class contains a single abstract method, `preprocess_function`, that subclasses must implement.
392 | - An optional tokenizer can be passed during initialization and used within the preprocessing method if needed.
393 | 
394 | #### Rules for extending:
395 | 1. **Mandatory Implementation**: Any class extending the `Preprocessor` must provide a concrete implementation of the `preprocess_function`.
396 | 2. **Method Signature**: The `preprocess_function` must have the same signature across all implementations: `(sample, padding="max_length")`.
397 | 3. **Return Type**: The `preprocess_function` should return a dictionary compatible with the transformer's model input. Typically, this includes tokenized input sequences and associated labels.
398 | 4. **Use Tokenizer Judiciously**: While the tokenizer is provided and can be used within the preprocess function, it's essential to remember that different tokenizers may have distinct properties and methods. Ensure compatibility.
399 | 5. **Ensure Padding Compatibility**: Since padding is a parameter, make sure to handle different padding strategies like `max_length`, `longest`, etc.
400 | 
401 | #### Example:
402 | 
403 | ```python
404 | from abc import ABC, abstractmethod
405 | 
406 | class Preprocessor(ABC):
407 | 
408 |     def __init__(self, tokenizer):
409 |         self.tokenizer = tokenizer
410 | 
411 |     @abstractmethod
412 |     def preprocess_function(self, sample, padding="max_length"):
413 |         pass
414 | ```
415 | 
416 | #### Usage:
417 | To use the `Preprocessor` class, follow the steps below:
418 | 
419 | 1. **Create a Custom Preprocessor**:
420 |     Extend the `Preprocessor` abstract class and implement the `preprocess_function` according to your requirements.
421 | 
422 |    ```python
423 |    class CustomPreprocessor(Preprocessor):
424 |        def preprocess_function(self, sample, padding="max_length"):
425 |            # Your custom preprocessing logic here
426 |            pass
427 |    ```
428 | 
429 | 2. **Pass to FineTuner**:
430 |    Instantiate your custom preprocessor and pass it to the `FineTuner` during initialization.
431 | 
432 |    ```python
433 |    custom_preprocessor = CustomPreprocessor(tokenizer=YourTokenizer)
434 |    finetuner = FineTuner(model_id='your_model_id', preprocessor=custom_preprocessor)
435 |    ```
436 | 
437 | By adhering to the outlined structure and rules, you ensure that custom preprocessing functions are easily integrated into the existing pipeline and remain compatible with the overall training and generation processes.
438 | 
439 | --- 
440 | 
441 | This documentation provides a concise overview, rules, and guidelines for effectively using and extending the `Preprocessor` abstract class.
442 | 
443 | ---
444 | 
445 | # Custom Training Logic
446 | ### 1. The `TrainerConfiguration` Abstract Class
447 | 
448 | Let's remove the specifics of the Lora config and collator, and instead provide one abstract method that allows for configuring the model and trainer.
449 | 
450 | ```python
451 | from abc import ABC, abstractmethod
452 | 
453 | class TrainerConfiguration(ABC):
454 | 
455 |     @abstractmethod
456 |     def configure(self, model, tokenizer, output_dir, num_train_epochs):
457 |         """Configures the model, collator, and training arguments.
458 |         
459 |         Returns:
460 |             tuple: (configured_model, data_collator, training_args)
461 |         """
462 |         pass
463 | ```
464 | 
465 | ### 2. Default Implementation
466 | 
467 | A simple default implementation can retain the previous configurations:
468 | 
469 | ```python
470 | class DefaultTrainerConfig(TrainerConfiguration):
471 |     
472 |     def configure(self, model, tokenizer, output_dir, num_train_epochs):
473 |         # LoraConfig (just as an example)
474 |         lora_config = LoraConfig(
475 |             r=16,
476 |             lora_alpha=32,
477 |             target_modules=["q", "v"],
478 |             bias="none",
479 |             task_type=TaskType.SEQ_2_SEQ_LM,
480 |         )
481 |         model = get_peft_model(model, lora_config)
482 | 
483 |         # DataCollator
484 |         data_collator = DataCollatorForSeq2Seq(tokenizer, model=model, label_pad_token_id=-100, pad_to_multiple_of=8)
485 | 
486 |         # Training arguments
487 |         training_args = Seq2SeqTrainingArguments(
488 |             output_dir=output_dir,
489 |             auto_find_batch_size=True,
490 |             learning_rate=1e-3,
491 |             num_train_epochs=num_train_epochs,
492 |             logging_dir=f"{output_dir}/logs",
493 |             logging_strategy="steps",
494 |             logging_steps=500,
495 |             save_strategy="no",
496 |             report_to="tensorboard"
497 |         )
498 | 
499 |         return model, data_collator, training_args
500 | ```
501 | 
502 | ### 3. Integration with `FineTuner`
503 | 
504 | Incorporate the `TrainerConfiguration` into `FineTuner`:
505 | 
506 | ```python
507 | class FineTuner:
508 |     
509 |     def __init__(self, model_id: str, device: str = None, dataset_name=None, trainer_config=None, ...):
510 |         ...
511 |         self.trainer_config = trainer_config if trainer_config else DefaultTrainerConfig()
512 | 
513 |     ...
514 | 
515 |     def train(self, output_dir, num_train_epochs):
516 |         self.model, data_collator, training_args = self.trainer_config.configure(self.model, self.tokenizer, output_dir, num_train_epochs)
517 |         
518 |         tokenized_dataset = self.preprocess_data(512, 150)
519 |         trainer = Seq2SeqTrainer(model=self.model, args=training_args, data_collator=data_collator, train_dataset=tokenized_dataset["train"])
520 |         trainer.train()
521 | ```
522 | 
523 | ---
524 | 
525 | ### Conclusion:
526 | 
527 | The `FineTuner` class in the `finetuning_suite` module of the `zeta` library facilitates the fine-tuning of pre-trained models for causal language modeling tasks using the Hugging Face `transformers` library. This class includes functionalities for data preprocessing, model training, and text generation. It also supports quantizing the model weights to reduce the model size and accelerate inference.


--------------------------------------------------------------------------------