├── .env.template ├── .gitignore ├── LICENSE ├── README.md ├── docker ├── cloud │ └── docker-compose.yml └── local │ └── docker-compose.yml ├── docs └── 2022-chevrolet-colorado-ebrochure.pdf ├── infra └── bicep │ ├── main.bicep │ └── openai.bicep ├── redis-openAI-doc-chat.ipynb └── requirements.txt /.env.template: -------------------------------------------------------------------------------- 1 | # General OpenAI Env Vars 2 | OPENAI_TEXT_MODEL=gpt-35-turbo 3 | OPENAI_EMBEDDING_MODEL=text-embedding-ada-002 4 | OPENAI_TEMPERATURE=0.7 5 | OPENAI_MAX_TOKENS=50 6 | 7 | # OpenAI Direct Env Vars 8 | OPENAI_API_KEY= 9 | OPENAI_API_BASE=https://api.openai.com/v1/ 10 | 11 | # Azure OpenAI Env Vars 12 | #OPENAI_API_VERSION=2023-05-15 # use OPENAI_API_VERSION only with Azure OpenAI 13 | AZURE_EMBED_MODEL_DEPLOYMENT_NAME= 14 | AZURE_TEXT_MODEL_DEPLOYMENT_NAME= 15 | AZURE_OPENAI_API_BASE=https://.openai.azure.com/ 16 | 17 | # General Env Vars 18 | CHUNK_SIZE=500 19 | CHUNK_OVERLAP=0.2 20 | 21 | # Redis Env Vars 22 | REDIS_HOST=redis 23 | REDIS_PORT=6379 24 | REDIS_PASSWORD= -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | docker-compose-keys.yml 2 | .ipynb_checkpoints 3 | .env 4 | .env-azure 5 | .env-azure-local 6 | .env-openai 7 | .local 8 | .cache 9 | .jupyter 10 | .ipython 11 | .npm -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Redis Inc 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Redis-LLM-Document-Chat 2 | 3 | This repository provides the materials for the joint Redis/Microsoft blog post [here](https://techcommunity.microsoft.com/t5/azure-developer-community-blog/vector-similarity-search-with-azure-cache-for-redis-enterprise/ba-p/3822059). It contains a Jupyter notebook that demonstrates how to use Redis as a vector database to store and retrieve document vectors. The notebook also shows how to use LlamaIndex to perform semantic search for context within documents 4 | and provide a chatbot-like experience using OpenAI. 5 | 6 | https://user-images.githubusercontent.com/13009163/237002719-26e3118d-77ee-4ded-96f5-6ba801cae66c.mov 7 | 8 | 9 | ## Getting Started 10 | 11 | The LLM Document Chat tutorial is **intended to run on a dockerized Jupyter lab environment** to ensure ideal experience and minimal Python environment hickups. At a minimum, you will need to install [Docker Desktop](https://www.docker.com/products/docker-desktop/) (*which comes with Docker Compose*) to run this example. 12 | 13 | The project maintains a `.env.template` with the following variables pre-defined: 14 | 15 | ```bash 16 | # General OpenAI Env Vars 17 | OPENAI_TEXT_MODEL=gpt-35-turbo 18 | OPENAI_EMBEDDING_MODEL=text-embedding-ada-002 19 | OPENAI_TEMPERATURE=0.7 20 | OPENAI_MAX_TOKENS=50 21 | 22 | # OpenAI Direct Env Vars 23 | OPENAI_API_KEY= 24 | OPENAI_API_BASE=https://api.openai.com/v1/ 25 | 26 | # Azure OpenAI Env Vars 27 | #OPENAI_API_VERSION=2023-05-15 # use OPENAI_API_VERSION only with Azure OpenAI 28 | AZURE_EMBED_MODEL_DEPLOYMENT_NAME= 29 | AZURE_TEXT_MODEL_DEPLOYMENT_NAME= 30 | AZURE_OPENAI_API_BASE=https://.openai.azure.com/ 31 | 32 | # General Env Vars 33 | CHUNK_SIZE=500 34 | CHUNK_OVERLAP=0.2 35 | 36 | # Redis Env Vars 37 | REDIS_HOST=redis 38 | REDIS_PORT=6379 39 | REDIS_PASSWORD= 40 | ``` 41 | 42 | Make a copy of this file to create `.env` as follows: 43 | ```bash 44 | $ cp .env.template .env 45 | ``` 46 | 47 | Update portions of the env file based on your choices below: 48 | 1. **[Choose your OpenAI provider](#choose-your-openai-provider)** 49 | 2. **[Choose your Redis provider](#choose-your-redis-provider)** 50 | 51 | 52 | --- 53 | **NOTE** If you are hosting this sample on Azure, you can quickly deploy Azure OpenAI and Azure Cache for Redis Enterprise instance with the right configurations by running the following commands: 54 | 55 | ```bash 56 | az group create -n LLMDocChatRG -l eastus 57 | 58 | az deployment group create --template-file infra\bicep\main.bicep -g LLMDocChatRG 59 | ``` 60 | 61 | To remove all Azure resources, simply delete the resource group using: 62 | ```bash 63 | az group delete -n LLMDocChatRG 64 | 65 | ``` 66 | 67 | Alternately, you can choose to use OpenAI or other Redis providers as documented below. 68 | --- 69 | 70 | ### Choose your OpenAI provider 71 | You can choose between **[Azure OpenAI Service](#to-use-azure-openai)** (fully hosted and managed by Azure) and **[OpenAI](#to-use-openai)** direct. 72 | 73 | 74 | #### To Use Azure OpenAI (Recommended) 75 | To use Azure OpenAI, you will need to follow these instructions 76 | 77 | 1. Create an Azure OpenAI resource. 78 | 2. Create model deployments for the text and embedding models. 79 | 4. Configure the ``.env`` file adding the specific values for your deployments. 80 | 81 | ```bash 82 | AZURE_EMBED_MODEL_DEPLOYMENT_NAME= 83 | AZURE_TEXT_MODEL_DEPLOYMENT_NAME= 84 | AZURE_OPENAI_API_BASE=https://.openai.azure.com/ 85 | ``` 86 | 87 | #### To Use OpenAI 88 | 89 | To use OpenAI, you will need to follow these instructions 90 | 91 | 1. Create an OpenAI account. 92 | 2. Create an OpenAI API key. 93 | 3. Configure the ``.env`` file adding the specific values for your deployments. 94 | 95 | ```bash 96 | OPENAI_API_KEY= 97 | ``` 98 | 99 | 100 | ### Choose your Redis provider 101 | The tutorial will require the usage of the Redis Search & Query features, including support for Vector Similarity Search. There are three options for running Redis: 102 | 103 | 1. [Redis Enterprise Cloud - Free Instance](https://redis.com/try-free) 104 | 2. Azure Redis Enterprise (ACRE) 105 | 3. Redis Stack (local docker) 106 | 107 | For each, the following options are **required** and must be updated in your local `.env` file: 108 | 109 | ```bash 110 | REDIS_PASSWORD= 111 | REDIS_HOST= 112 | REDIS_PORT= 113 | ``` 114 | 115 | There are some ``docker-compose.yml`` files in the ``docker`` directory that will help spin up 116 | redis-stack locally and redisinsight in the case where a remote Redis is being used (like ACRE). 117 | 118 | ## Run 119 | 120 | To open the jupyter environment through docker, follow these steps: 121 | 122 | 1. Clone this repository to your local machine. 123 | 2. Copy the ``.env.template`` to ``.env`` and configure the values as outlined above. 124 | 3. Run with Docker Compose: 125 | 126 | **For Cloud or Azure Redis Enterprise** 127 | ```bash 128 | docker compose -f docker/cloud/docker-compose.yml up 129 | ``` 130 | 131 | **For Local (Docker) Redis Stack** 132 | ```bash 133 | docker compose -f docker/local/docker-compose.yml up 134 | ``` 135 | 4. Open the Jupyter lab session in your browser at `http://127.0.0.1:8888/lab?token={YOUR GENERATED TOKEN}`. *Check the terminal logs for the token string.* -------------------------------------------------------------------------------- /docker/cloud/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "3.9" 2 | services: 3 | redisinsight: 4 | image: redislabs/redisinsight:latest 5 | ports: 6 | - 8001:8001 7 | restart: always 8 | 9 | jupyter: 10 | image: jupyter/minimal-notebook:latest 11 | container_name: jupyter 12 | volumes: 13 | - ../../:/home/jovyan/ 14 | env_file: 15 | - ../../.env 16 | ports: 17 | - 8888:8888 18 | depends_on: 19 | - "redisinsight" 20 | environment: 21 | JUPYTER_ENABLE_LAB: "yes" 22 | 23 | volumes: 24 | redisinsight_data: 25 | -------------------------------------------------------------------------------- /docker/local/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "3.9" 2 | 3 | services: 4 | redis: 5 | image: redis/redis-stack:latest 6 | ports: 7 | - "6379:6379" 8 | - "8001:8001" 9 | volumes: 10 | - redis_data:/data 11 | healthcheck: 12 | test: ["CMD", "redis-cli", "-h", "localhost", "-p", "6379", "ping"] 13 | interval: 2s 14 | timeout: 1m30s 15 | retries: 5 16 | start_period: 5s 17 | jupyter: 18 | image: jupyter/minimal-notebook:latest 19 | container_name: jupyter 20 | volumes: 21 | - ../../:/home/jovyan/ 22 | env_file: 23 | - ../../.env 24 | ports: 25 | - 8888:8888 26 | depends_on: 27 | - "redis" 28 | environment: 29 | JUPYTER_ENABLE_LAB: "yes" 30 | 31 | volumes: 32 | redis_data: -------------------------------------------------------------------------------- /docs/2022-chevrolet-colorado-ebrochure.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/redis-developer/LLM-Document-Chat/4468f7b5b243e7fdeb36a8e53a75efc77e89a92b/docs/2022-chevrolet-colorado-ebrochure.pdf -------------------------------------------------------------------------------- /infra/bicep/main.bicep: -------------------------------------------------------------------------------- 1 | @description('That name is the name of our application. It has to be unique.Type a name followed by your resource group name. (-)') 2 | param cognitiveServiceName string = 'CognitiveService' 3 | 4 | @description('Location for all resources.') 5 | param location string = resourceGroup().location 6 | param OpenAIQuota int = 100 7 | 8 | module openAi 'openai.bicep' = { 9 | name: 'openai' 10 | params: { 11 | name: cognitiveServiceName 12 | location: location 13 | sku: { 14 | name: 'S0' 15 | 16 | } 17 | deployments: [ 18 | { 19 | name: 'EmbeddingAdaModel' 20 | model: { 21 | format: 'OpenAI' 22 | name: 'text-embedding-ada-002' 23 | version: '2' 24 | } 25 | capacity: OpenAIQuota 26 | } 27 | { 28 | name: 'GPT35Model' 29 | model: { 30 | format: 'OpenAI' 31 | name: 'gpt-35-turbo-instruct' 32 | } 33 | sku: { 34 | name: 'Standard' 35 | capacity: OpenAIQuota 36 | } 37 | } 38 | ] 39 | } 40 | } 41 | 42 | param redisEnterpriseName string = 'RedisEnterpriseLLMDocChat' 43 | resource RedisEnterprise 'Microsoft.Cache/redisEnterprise@2022-01-01' = { 44 | name: redisEnterpriseName 45 | location: location 46 | sku: { 47 | capacity: 2 48 | name: 'Enterprise_E5' 49 | } 50 | properties: { 51 | minimumTlsVersion: '1.2' 52 | } 53 | } 54 | 55 | resource RedisEnterpriseDB 'Microsoft.Cache/redisEnterprise/databases@2022-01-01' = { 56 | name: 'default' 57 | parent: RedisEnterprise 58 | properties: { 59 | clientProtocol: 'PlainText' 60 | clusteringPolicy: 'EnterpriseCluster' 61 | evictionPolicy: 'NoEviction' 62 | modules: [ 63 | { 64 | name: 'RediSearch' 65 | } 66 | ] 67 | } 68 | } 69 | 70 | output REDIS_ADDRESS string = RedisEnterprise.properties.hostName 71 | -------------------------------------------------------------------------------- /infra/bicep/openai.bicep: -------------------------------------------------------------------------------- 1 | param name string 2 | param location string = resourceGroup().location 3 | param tags object = {} 4 | 5 | param customSubDomainName string = name 6 | param deployments array = [] 7 | param kind string = 'OpenAI' 8 | param publicNetworkAccess string = 'Enabled' 9 | param sku object = { 10 | name: 'S0' 11 | } 12 | 13 | resource account 'Microsoft.CognitiveServices/accounts@2023-05-01' = { 14 | name: name 15 | location: location 16 | tags: tags 17 | kind: kind 18 | properties: { 19 | customSubDomainName: customSubDomainName 20 | publicNetworkAccess: publicNetworkAccess 21 | } 22 | sku: sku 23 | } 24 | 25 | @batchSize(1) 26 | resource deployment 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = [for deployment in deployments: { 27 | parent: account 28 | name: deployment.name 29 | properties: { 30 | model: deployment.model 31 | raiPolicyName: contains(deployment, 'raiPolicyName') ? deployment.raiPolicyName : null 32 | } 33 | sku: contains(deployment, 'sku') ? deployment.sku : { 34 | name: 'Standard' 35 | capacity: deployment.capacity 36 | } 37 | }] 38 | 39 | output endpoint string = account.properties.endpoint 40 | output id string = account.id 41 | output name string = account.name 42 | -------------------------------------------------------------------------------- /redis-openAI-doc-chat.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "attachments": {}, 5 | "cell_type": "markdown", 6 | "id": "0b692c73", 7 | "metadata": {}, 8 | "source": [ 9 | "# Using Redis and OpenAI to chat with PDF documents\n", 10 | "\n", 11 | "This notebook demonstrates how to use RedisAI and (Azure) OpenAI to chat with PDF documents. The PDF included is\n", 12 | "a informational brochure about the Chevy Colorado pickup truck.\n", 13 | "\n", 14 | "In this notebook, we will use LLamaIndex to chunk, vectorize, and store the PDF document in Redis as vectors\n", 15 | "alongside associated text. The query interface provided by LLamaIndex will be used to search for relevant\n", 16 | "information given queries from the user." 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": 4, 22 | "id": "949e6cf1", 23 | "metadata": { 24 | "scrolled": true, 25 | "tags": [] 26 | }, 27 | "outputs": [ 28 | { 29 | "name": "stdout", 30 | "output_type": "stream", 31 | "text": [ 32 | "Requirement already satisfied: pypdf in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (3.17.0)\n", 33 | "Requirement already satisfied: PyPDF2 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (3.0.1)\n", 34 | "Requirement already satisfied: transformers in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (4.35.0)\n", 35 | "Requirement already satisfied: tiktoken in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (0.5.1)\n", 36 | "Requirement already satisfied: ipywidgets in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (8.1.1)\n", 37 | "Requirement already satisfied: nltk in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (3.8.1)\n", 38 | "Requirement already satisfied: redis>=5.0.0 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (5.0.1)\n", 39 | "Requirement already satisfied: llama_index==0.8.58 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (0.8.58)\n", 40 | "Collecting openai==0.28.1 (from -r requirements.txt (line 10))\n", 41 | " Downloading openai-0.28.1-py3-none-any.whl.metadata (11 kB)\n", 42 | "Requirement already satisfied: SQLAlchemy>=1.4.49 in /opt/conda/lib/python3.11/site-packages (from SQLAlchemy[asyncio]>=1.4.49->llama_index==0.8.58->-r requirements.txt (line 9)) (2.0.22)\n", 43 | "Requirement already satisfied: aiostream<0.6.0,>=0.5.2 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.5.2)\n", 44 | "Requirement already satisfied: dataclasses-json<0.6.0,>=0.5.7 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.5.14)\n", 45 | "Requirement already satisfied: deprecated>=1.2.9.3 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.2.14)\n", 46 | "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (2023.10.0)\n", 47 | "Requirement already satisfied: langchain>=0.0.303 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.0.331)\n", 48 | "Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.5.8)\n", 49 | "Requirement already satisfied: numpy in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.26.1)\n", 50 | "Requirement already satisfied: pandas in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (2.1.2)\n", 51 | "Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (8.2.3)\n", 52 | "Requirement already satisfied: typing-extensions>=4.5.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (4.8.0)\n", 53 | "Requirement already satisfied: typing-inspect>=0.8.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.9.0)\n", 54 | "Requirement already satisfied: urllib3<2 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.26.18)\n", 55 | "Requirement already satisfied: requests>=2.20 in /opt/conda/lib/python3.11/site-packages (from openai==0.28.1->-r requirements.txt (line 10)) (2.31.0)\n", 56 | "Requirement already satisfied: tqdm in /opt/conda/lib/python3.11/site-packages (from openai==0.28.1->-r requirements.txt (line 10)) (4.66.1)\n", 57 | "Requirement already satisfied: aiohttp in /opt/conda/lib/python3.11/site-packages (from openai==0.28.1->-r requirements.txt (line 10)) (3.8.6)\n", 58 | "Requirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (3.13.1)\n", 59 | "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (0.17.3)\n", 60 | "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (23.2)\n", 61 | "Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (6.0.1)\n", 62 | "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (2023.10.3)\n", 63 | "Requirement already satisfied: tokenizers<0.15,>=0.14 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (0.14.1)\n", 64 | "Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (0.4.0)\n", 65 | "Requirement already satisfied: comm>=0.1.3 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (0.1.4)\n", 66 | "Requirement already satisfied: ipython>=6.1.0 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (8.16.1)\n", 67 | "Requirement already satisfied: traitlets>=4.3.1 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (5.11.2)\n", 68 | "Requirement already satisfied: widgetsnbextension~=4.0.9 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (4.0.9)\n", 69 | "Requirement already satisfied: jupyterlab-widgets~=3.0.9 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (3.0.9)\n", 70 | "Requirement already satisfied: click in /opt/conda/lib/python3.11/site-packages (from nltk->-r requirements.txt (line 7)) (8.1.7)\n", 71 | "Requirement already satisfied: joblib in /opt/conda/lib/python3.11/site-packages (from nltk->-r requirements.txt (line 7)) (1.3.2)\n", 72 | "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /opt/conda/lib/python3.11/site-packages (from dataclasses-json<0.6.0,>=0.5.7->llama_index==0.8.58->-r requirements.txt (line 9)) (3.20.1)\n", 73 | "Requirement already satisfied: wrapt<2,>=1.10 in /opt/conda/lib/python3.11/site-packages (from deprecated>=1.2.9.3->llama_index==0.8.58->-r requirements.txt (line 9)) (1.15.0)\n", 74 | "Requirement already satisfied: backcall in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.2.0)\n", 75 | "Requirement already satisfied: decorator in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (5.1.1)\n", 76 | "Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.19.1)\n", 77 | "Requirement already satisfied: matplotlib-inline in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.1.6)\n", 78 | "Requirement already satisfied: pickleshare in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.7.5)\n", 79 | "Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (3.0.39)\n", 80 | "Requirement already satisfied: pygments>=2.4.0 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (2.16.1)\n", 81 | "Requirement already satisfied: stack-data in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.6.2)\n", 82 | "Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (4.8.0)\n", 83 | "Requirement already satisfied: anyio<4.0 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (3.7.1)\n", 84 | "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (1.33)\n", 85 | "Requirement already satisfied: langsmith<0.1.0,>=0.0.52 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (0.0.57)\n", 86 | "Requirement already satisfied: pydantic<3,>=1 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (2.4.2)\n", 87 | "Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (23.1.0)\n", 88 | "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (3.3.0)\n", 89 | "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (6.0.4)\n", 90 | "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (4.0.3)\n", 91 | "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (1.9.2)\n", 92 | "Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (1.4.0)\n", 93 | "Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (1.3.1)\n", 94 | "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests>=2.20->openai==0.28.1->-r requirements.txt (line 10)) (3.4)\n", 95 | "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests>=2.20->openai==0.28.1->-r requirements.txt (line 10)) (2023.7.22)\n", 96 | "Requirement already satisfied: greenlet!=0.4.17 in /opt/conda/lib/python3.11/site-packages (from SQLAlchemy>=1.4.49->SQLAlchemy[asyncio]>=1.4.49->llama_index==0.8.58->-r requirements.txt (line 9)) (3.0.0)\n", 97 | "Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.11/site-packages (from typing-inspect>=0.8.0->llama_index==0.8.58->-r requirements.txt (line 9)) (1.0.0)\n", 98 | "Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.11/site-packages (from pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (2.8.2)\n", 99 | "Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.11/site-packages (from pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (2023.3.post1)\n", 100 | "Requirement already satisfied: tzdata>=2022.1 in /opt/conda/lib/python3.11/site-packages (from pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (2023.3)\n", 101 | "Requirement already satisfied: sniffio>=1.1 in /opt/conda/lib/python3.11/site-packages (from anyio<4.0->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (1.3.0)\n", 102 | "Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/conda/lib/python3.11/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.8.3)\n", 103 | "Requirement already satisfied: jsonpointer>=1.9 in /opt/conda/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (2.4)\n", 104 | "Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.11/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.7.0)\n", 105 | "Requirement already satisfied: wcwidth in /opt/conda/lib/python3.11/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.2.8)\n", 106 | "Requirement already satisfied: annotated-types>=0.4.0 in /opt/conda/lib/python3.11/site-packages (from pydantic<3,>=1->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (0.6.0)\n", 107 | "Requirement already satisfied: pydantic-core==2.10.1 in /opt/conda/lib/python3.11/site-packages (from pydantic<3,>=1->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (2.10.1)\n", 108 | "Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (1.16.0)\n", 109 | "Requirement already satisfied: executing>=1.2.0 in /opt/conda/lib/python3.11/site-packages (from stack-data->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (1.2.0)\n", 110 | "Requirement already satisfied: asttokens>=2.1.0 in /opt/conda/lib/python3.11/site-packages (from stack-data->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (2.4.0)\n", 111 | "Requirement already satisfied: pure-eval in /opt/conda/lib/python3.11/site-packages (from stack-data->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.2.2)\n", 112 | "Downloading openai-0.28.1-py3-none-any.whl (76 kB)\n", 113 | "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.0/77.0 kB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", 114 | "\u001b[?25hInstalling collected packages: openai\n", 115 | " Attempting uninstall: openai\n", 116 | " Found existing installation: openai 1.0.1\n", 117 | " Uninstalling openai-1.0.1:\n", 118 | " Successfully uninstalled openai-1.0.1\n", 119 | "Successfully installed openai-0.28.1\n", 120 | "Note: you may need to restart the kernel to use updated packages.\n" 121 | ] 122 | } 123 | ], 124 | "source": [ 125 | "# Install the Python requirements\n", 126 | "%pip install -r requirements.txt --no-cache-dir" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": 1, 132 | "id": "47264e32", 133 | "metadata": { 134 | "ExecuteTime": { 135 | "end_time": "2023-02-10T12:20:23.988789Z", 136 | "start_time": "2023-02-10T12:20:23.967877Z" 137 | }, 138 | "tags": [] 139 | }, 140 | "outputs": [], 141 | "source": [ 142 | "import os\n", 143 | "import sys\n", 144 | "import logging\n", 145 | "\n", 146 | "logging.basicConfig(\n", 147 | " stream=sys.stdout, level=logging.INFO\n", 148 | ") # logging.DEBUG for more verbose output\n", 149 | "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n", 150 | "\n", 151 | "import textwrap\n", 152 | "import openai\n", 153 | "\n", 154 | "from langchain.llms import AzureOpenAI, OpenAI\n", 155 | "from langchain.embeddings import OpenAIEmbeddings\n", 156 | "from llama_index.vector_stores import RedisVectorStore\n", 157 | "from llama_index import LangchainEmbedding\n", 158 | "from llama_index import (\n", 159 | " GPTVectorStoreIndex,\n", 160 | " SimpleDirectoryReader,\n", 161 | " LLMPredictor,\n", 162 | " PromptHelper,\n", 163 | " ServiceContext,\n", 164 | " StorageContext\n", 165 | ")" 166 | ] 167 | }, 168 | { 169 | "attachments": {}, 170 | "cell_type": "markdown", 171 | "id": "0ad91218", 172 | "metadata": {}, 173 | "source": [ 174 | "# Azure OpenAI | OpenAI (direct)\n", 175 | "\n", 176 | "The notebook allows the user two choose between using the OpenAI and Azure OpenAI endpoints. Make sure to follow the instructions in the README and set the .env correctly according to whichever API you are using. \n", 177 | "\n", 178 | "NOTE: ONLY ONE API CAN BE USED AT A TIME.\n", 179 | "\n", 180 | "- **[Use Azure OpenAI](#Azure-OpenAI)**\n", 181 | "- **[Use OpenAI](#OpenAI) (direct)**" 182 | ] 183 | }, 184 | { 185 | "attachments": {}, 186 | "cell_type": "markdown", 187 | "id": "0023333d", 188 | "metadata": {}, 189 | "source": [ 190 | "## Azure OpenAI \n", 191 | "\n", 192 | "Here we setup the AzureOpenAI models and API keys that we set by reading from the environment above. The ``PromptHelper`` sets the parameters for the OpenAI model. The classes defined here are used together to provide a QnA interface between the user and the LLM." 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": 2, 198 | "id": "32a77108", 199 | "metadata": {}, 200 | "outputs": [ 201 | { 202 | "name": "stdout", 203 | "output_type": "stream", 204 | "text": [ 205 | "Using OpenAI models: text-embedding-ada-002 and gpt-35-turbo\n", 206 | "Using Azure deployments: and \n" 207 | ] 208 | } 209 | ], 210 | "source": [ 211 | "# setup Llama Index to use Azure OpenAI\n", 212 | "openai.api_type = \"azure\"\n", 213 | "openai.api_base = os.getenv(\"AZURE_OPENAI_API_BASE\")\n", 214 | "openai.api_version = os.getenv(\"OPENAI_API_VERSION\")\n", 215 | "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n", 216 | "\n", 217 | "\n", 218 | "# Get the OpenAI model names ex. \"text-embedding-ada-002\"\n", 219 | "embedding_model = os.getenv(\"OPENAI_EMBEDDING_MODEL\")\n", 220 | "text_model = os.getenv(\"OPENAI_TEXT_MODEL\")\n", 221 | "# get the Azure Deployment name for the model\n", 222 | "embedding_model_deployment = os.getenv(\"AZURE_EMBED_MODEL_DEPLOYMENT_NAME\")\n", 223 | "text_model_deployment = os.getenv(\"AZURE_TEXT_MODEL_DEPLOYMENT_NAME\")\n", 224 | "\n", 225 | "print(f\"Using OpenAI models: {embedding_model} and {text_model}\")\n", 226 | "print(f\"Using Azure deployments: {embedding_model_deployment} and {text_model_deployment}\")\n" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": 5, 232 | "id": "c67d58db", 233 | "metadata": {}, 234 | "outputs": [], 235 | "source": [ 236 | "\n", 237 | "llm = AzureOpenAI(deployment_name=text_model_deployment, model_kwargs={\n", 238 | " \"api_key\": openai.api_key,\n", 239 | " \"api_base\": openai.api_base,\n", 240 | " \"api_type\": openai.api_type,\n", 241 | " \"api_version\": openai.api_version,\n", 242 | "})\n", 243 | "llm_predictor = LLMPredictor(llm=llm)\n", 244 | "\n", 245 | "embedding_llm = LangchainEmbedding(\n", 246 | " OpenAIEmbeddings(\n", 247 | " model=embedding_model,\n", 248 | " deployment=embedding_model_deployment,\n", 249 | " openai_api_key= openai.api_key,\n", 250 | " openai_api_base=openai.api_base,\n", 251 | " openai_api_type=openai.api_type,\n", 252 | " openai_api_version=openai.api_version,\n", 253 | " ),\n", 254 | " embed_batch_size=1,\n", 255 | ")" 256 | ] 257 | }, 258 | { 259 | "attachments": {}, 260 | "cell_type": "markdown", 261 | "id": "a31b8dae", 262 | "metadata": {}, 263 | "source": [ 264 | "## OpenAI\n", 265 | "\n", 266 | "The ``OpenAI`` class provides a simple interface to the OpenAI API.\n" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": 3, 272 | "id": "4ce97716", 273 | "metadata": {}, 274 | "outputs": [ 275 | { 276 | "name": "stdout", 277 | "output_type": "stream", 278 | "text": [ 279 | "Using OpenAI models: text-embedding-ada-002 and gpt-35-turbo\n" 280 | ] 281 | } 282 | ], 283 | "source": [ 284 | "# setup Llama Index to use OpenAI direct\n", 285 | "openai.api_type = \"openai\"\n", 286 | "openai.api_version = os.getenv(\"OPENAI_API_VERSION\")\n", 287 | "openai.api_base = os.getenv(\"OPENAI_API_BASE\")\n", 288 | "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n", 289 | "\n", 290 | "# Get the OpenAI model names ex. \"text-embedding-ada-002\"\n", 291 | "embedding_model = os.getenv(\"OPENAI_EMBEDDING_MODEL\")\n", 292 | "text_model = os.getenv(\"OPENAI_TEXT_MODEL\")\n", 293 | "\n", 294 | "print(f\"Using OpenAI models: {embedding_model} and {text_model}\")" 295 | ] 296 | }, 297 | { 298 | "cell_type": "code", 299 | "execution_count": 4, 300 | "id": "076cf33f", 301 | "metadata": {}, 302 | "outputs": [], 303 | "source": [ 304 | "# Set up LLM\n", 305 | "llm = OpenAI(model_kwargs={\n", 306 | " \"api_key\": openai.api_key,\n", 307 | " \"api_base\": openai.api_base,\n", 308 | " \"api_type\": openai.api_type,\n", 309 | " \"api_version\" : openai.api_version,\n", 310 | "})\n", 311 | "llm_predictor = LLMPredictor(llm=llm)\n", 312 | "\n", 313 | "# Set up Embedding model\n", 314 | "embedding_llm = LangchainEmbedding(\n", 315 | " OpenAIEmbeddings(\n", 316 | " model=embedding_model,\n", 317 | " openai_api_version=openai.api_version,\n", 318 | " openai_api_key= openai.api_key,\n", 319 | " openai_api_base=openai.api_base,\n", 320 | " openai_api_type=openai.api_type,\n", 321 | " ),\n", 322 | " embed_batch_size=1,\n", 323 | ")" 324 | ] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "id": "59ff935d", 329 | "metadata": {}, 330 | "source": [ 331 | "### LLamaIndex\n", 332 | "\n", 333 | "[LlamaIndex](https://github.com/jerryjliu/llama_index) (GPT Index) is a project that provides a central interface to connect your LLM's with external data sources. It provides a simple interface to vectorize and store embeddings in Redis, create search indices using Redis, and perform vector search to find context for generative models like GPT.\n", 334 | "\n", 335 | "Here we will use it to load in the documents (Chevy Colorado Brochure)." 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": 5, 341 | "id": "68cbd239-880e-41a3-98d8-dbb3fab55431", 342 | "metadata": { 343 | "ExecuteTime": { 344 | "end_time": "2023-02-10T12:20:30.175678Z", 345 | "start_time": "2023-02-10T12:20:30.172456Z" 346 | }, 347 | "pycharm": { 348 | "is_executing": true 349 | }, 350 | "tags": [] 351 | }, 352 | "outputs": [ 353 | { 354 | "name": "stdout", 355 | "output_type": "stream", 356 | "text": [ 357 | "Document ID: c325c82f-9408-42fe-9a5a-c58e196ef5a3\n" 358 | ] 359 | } 360 | ], 361 | "source": [ 362 | "# load documents\n", 363 | "documents = SimpleDirectoryReader('./docs').load_data()\n", 364 | "print('Document ID:', documents[0].doc_id)" 365 | ] 366 | }, 367 | { 368 | "attachments": {}, 369 | "cell_type": "markdown", 370 | "id": "697a59d2", 371 | "metadata": {}, 372 | "source": [ 373 | "Llamaindex also works with frameworks like langchain to make prompting and other aspects of a chat based application easier. Here we can use the ``PromptHelper`` class to help us generate prompts for the (Azure) OpenAI model. The will be off by default as it can be tricky to setup correctly." 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": 6, 379 | "id": "147e7678", 380 | "metadata": {}, 381 | "outputs": [], 382 | "source": [ 383 | "# set number of output tokens\n", 384 | "num_output = int(os.getenv(\"OPENAI_MAX_TOKENS\"))\n", 385 | "# max LLM token input size\n", 386 | "max_input_size = int(os.getenv(\"CHUNK_SIZE\"))\n", 387 | "# set maximum chunk overlap\n", 388 | "max_chunk_overlap = float(os.getenv(\"CHUNK_OVERLAP\"))\n", 389 | "\n", 390 | "prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)" 391 | ] 392 | }, 393 | { 394 | "cell_type": "code", 395 | "execution_count": 7, 396 | "id": "132b7b01", 397 | "metadata": {}, 398 | "outputs": [ 399 | { 400 | "name": "stderr", 401 | "output_type": "stream", 402 | "text": [ 403 | "[nltk_data] Downloading package punkt to /tmp/llama_index...\n", 404 | "[nltk_data] Unzipping tokenizers/punkt.zip.\n" 405 | ] 406 | } 407 | ], 408 | "source": [ 409 | "# define the service we will use to answer questions\n", 410 | "# if you executive the Azure OpenAI code above, your Azure Models and creds will be used and the same for OpenAI\n", 411 | "service_context = ServiceContext.from_defaults(\n", 412 | " llm_predictor=llm_predictor,\n", 413 | " embed_model=embedding_llm,\n", 414 | " prompt_helper=prompt_helper # uncomment to use prompt_helper.\n", 415 | ")" 416 | ] 417 | }, 418 | { 419 | "attachments": {}, 420 | "cell_type": "markdown", 421 | "id": "dd270925", 422 | "metadata": {}, 423 | "source": [ 424 | "## Initialize Redis as a Vector Database\n", 425 | "\n", 426 | "Now we have our documents read in, we can initialize the ``RedisVectorStore``. This will allow us to store our vectors in Redis and create an index.\n", 427 | "\n", 428 | "The ``GPTVectorStoreIndex`` will then create the embeddings from the text chunks by calling out to OpenAI's API. The embeddings will be stored in Redis and an index will be created.\n", 429 | "\n", 430 | "NOTE: If you didn't set the ``OPENAI_API_KEY`` environment variable, you will get an error here." 431 | ] 432 | }, 433 | { 434 | "cell_type": "code", 435 | "execution_count": 8, 436 | "id": "fd85f591-334a-492a-ba80-89fe7c79288e", 437 | "metadata": {}, 438 | "outputs": [ 439 | { 440 | "name": "stdout", 441 | "output_type": "stream", 442 | "text": [ 443 | "Using Redis address: redis://:0kYQxoG2RfkL0zVzQRXm0MSwBmVJ3D89+csfCH8E4fo=@sgn-llm.eastus2.redisenterprise.cache.azure.net:10000\n" 444 | ] 445 | } 446 | ], 447 | "source": [ 448 | "def format_redis_conn_from_env(using_ssl=False):\n", 449 | " start = \"rediss://\" if using_ssl else \"redis://\"\n", 450 | " # if using RBAC\n", 451 | " password = os.getenv(\"REDIS_PASSWORD\", None)\n", 452 | " username = os.getenv(\"REDIS_USERNAME\", \"\")\n", 453 | " if password != None:\n", 454 | " start += f\"{username}:{password}@\"\n", 455 | "\n", 456 | " return start + f\"{os.getenv('REDIS_HOST')}:{os.getenv('REDIS_PORT')}\"\n", 457 | "\n", 458 | "# make using_ssl=True to use SSL with ACRE\n", 459 | "redis_url = format_redis_conn_from_env(using_ssl=False)\n", 460 | "print(f\"Using Redis address: {redis_url}\")\n" 461 | ] 462 | }, 463 | { 464 | "cell_type": "code", 465 | "execution_count": 9, 466 | "id": "30b0d59f-a7ac-413e-ab1f-3e46628d6e95", 467 | "metadata": {}, 468 | "outputs": [ 469 | { 470 | "data": { 471 | "text/plain": [ 472 | "True" 473 | ] 474 | }, 475 | "execution_count": 9, 476 | "metadata": {}, 477 | "output_type": "execute_result" 478 | } 479 | ], 480 | "source": [ 481 | "# Create VectorStore\n", 482 | "vector_store = RedisVectorStore(\n", 483 | " index_name=\"chevy_docs\",\n", 484 | " index_prefix=\"blog\",\n", 485 | " redis_url=redis_url,\n", 486 | " overwrite=True\n", 487 | ")\n", 488 | "\n", 489 | "# access the underlying client in the RedisVectorStore implementation to ping the redis instance\n", 490 | "vector_store.client.ping()" 491 | ] 492 | }, 493 | { 494 | "cell_type": "code", 495 | "execution_count": 10, 496 | "id": "ba1558b3", 497 | "metadata": { 498 | "ExecuteTime": { 499 | "end_time": "2023-02-10T12:20:33.735897Z", 500 | "start_time": "2023-02-10T12:20:30.404245Z" 501 | }, 502 | "pycharm": { 503 | "is_executing": true 504 | }, 505 | "tags": [] 506 | }, 507 | "outputs": [ 508 | { 509 | "name": "stdout", 510 | "output_type": "stream", 511 | "text": [ 512 | "INFO:llama_index.vector_stores.redis:Creating index chevy_docs\n", 513 | "Creating index chevy_docs\n", 514 | "INFO:llama_index.vector_stores.redis:Added 27 documents to index chevy_docs\n", 515 | "Added 27 documents to index chevy_docs\n" 516 | ] 517 | } 518 | ], 519 | "source": [ 520 | "storage_context = StorageContext.from_defaults(vector_store=vector_store)\n", 521 | "index = GPTVectorStoreIndex.from_documents(\n", 522 | " documents,\n", 523 | " storage_context=storage_context,\n", 524 | " service_context=service_context\n", 525 | ")" 526 | ] 527 | }, 528 | { 529 | "cell_type": "markdown", 530 | "id": "04304299-fc3e-40a0-8600-f50c3292767e", 531 | "metadata": {}, 532 | "source": [ 533 | "## Test the RAG pipeline!\n", 534 | "\n", 535 | "Now that we have our document stored in the index, we can ask questions against the index. The index will use the data stored in itself as the knowledge base for the LLM." 536 | ] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "execution_count": 11, 541 | "id": "35369eda", 542 | "metadata": { 543 | "ExecuteTime": { 544 | "end_time": "2023-02-10T12:20:51.328762Z", 545 | "start_time": "2023-02-10T12:20:33.822688Z" 546 | }, 547 | "tags": [] 548 | }, 549 | "outputs": [ 550 | { 551 | "name": "stdout", 552 | "output_type": "stream", 553 | "text": [ 554 | "INFO:llama_index.vector_stores.redis:Using filters: *\n", 555 | "Using filters: *\n", 556 | "INFO:llama_index.vector_stores.redis:Querying index chevy_docs\n", 557 | "Querying index chevy_docs\n", 558 | "INFO:llama_index.vector_stores.redis:Found 2 results for query with id ['blog/vector_bd662262-87bf-4428-9e3e-155d20e99622', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n", 559 | "Found 2 results for query with id ['blog/vector_bd662262-87bf-4428-9e3e-155d20e99622', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n", 560 | "\n", 561 | " The Chevrolet Colorado is available in Extended Cab, Crew Cab Short Box, and Crew Cab Long Box\n", 562 | "variants. It includes a 2.5L DOHC 4-cylinder with Variable Valve Timing (VVT) and Direct Injection,\n", 563 | "and a 3.6L DOHC V6 with Variable Valve Timing (VVT) and Direct Injection (Crew Cab 4x4 and Crew Cab\n", 564 | "Long Box 2WD models). The transmission is either a 6-speed automatic, electronically controlled with\n", 565 | "overdrive or an 8-speed automatic, electronically controlled with overdrive, Tow/Haul mode and Hitch\n", 566 | "Guidance 1 (Crew Cab 4x4 and Crew Cab Long Box 2WD models). It also includes 17\" Blade Silver\n", 567 | "Metallic-Painted Aluminum Wheels (standard on LT), 18\" Dark Argent Metallic-Painted Aluminum Wheels\n", 568 | "(available on LT), and 18\" Black-Painted Aluminum Wheels with Red Accents (available on LT with\n", 569 | "Redline Edition). Additional features available for the Chevrolet Colorado include Five AEV hot-\n", 570 | "stamped boron steel skid plates, AEV fender flares, Fog lamps, Front and rear floor liners with AEV\n", 571 | "logo, AEV embroidered head restraints, recovery points, and the compatibility with Apple CarPlay.\n" 572 | ] 573 | } 574 | ], 575 | "source": [ 576 | "query_engine = index.as_query_engine()\n", 577 | "response = query_engine.query(\"What types of variants are available for the Chevrolet Colorado?\")\n", 578 | "print(\"\\n\", textwrap.fill(str(response), 100))" 579 | ] 580 | }, 581 | { 582 | "cell_type": "code", 583 | "execution_count": 12, 584 | "id": "99212d33", 585 | "metadata": { 586 | "ExecuteTime": { 587 | "end_time": "2023-02-10T12:21:10.337294Z", 588 | "start_time": "2023-02-10T12:20:51.338718Z" 589 | }, 590 | "tags": [] 591 | }, 592 | "outputs": [ 593 | { 594 | "name": "stdout", 595 | "output_type": "stream", 596 | "text": [ 597 | "INFO:llama_index.vector_stores.redis:Using filters: *\n", 598 | "Using filters: *\n", 599 | "INFO:llama_index.vector_stores.redis:Querying index chevy_docs\n", 600 | "Querying index chevy_docs\n", 601 | "INFO:llama_index.vector_stores.redis:Found 2 results for query with id ['blog/vector_efcf4237-6811-4d27-b6b3-49f0062880f9', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n", 602 | "Found 2 results for query with id ['blog/vector_efcf4237-6811-4d27-b6b3-49f0062880f9', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n", 603 | "\n", 604 | " The Chevy Colorado's maximum towing capacity is 7,700 lbs with the available GM-exclusive Duramax®\n", 605 | "2.8L Turbo-Diesel engine and the Trailering Package, LT Convenience Package and Safety Package. The\n", 606 | "Colorado Crew Cab ZR2 also features an AEV-designed aluminum wheels, bumper with winch provisions,\n", 607 | "rear bumper with recovery points, five hot-stamped boron steel skid plates, fender flares, fog\n", 608 | "lamps, front and rear floor liners with AEV logo, and AEV embroidered head restraints. However, due\n", 609 | "to current supply chain shortages, certain features shown may have limited or late availability, or\n", 610 | "may no longer be available. The trailering capacity of your specific vehicle may vary depending on\n", 611 | "the weight of passengers, cargo and options or accessories. You can check which smartphones are\n", 612 | "compatible with the Vehicle User Interface at Google Play or Apple's App Store.\n" 613 | ] 614 | } 615 | ], 616 | "source": [ 617 | "response = query_engine.query(\"What is the maximum towing capacity of the chevy colorado?\")\n", 618 | "print(\"\\n\", textwrap.fill(str(response), 100))" 619 | ] 620 | }, 621 | { 622 | "cell_type": "code", 623 | "execution_count": 13, 624 | "id": "4a028452", 625 | "metadata": { 626 | "tags": [] 627 | }, 628 | "outputs": [ 629 | { 630 | "name": "stdout", 631 | "output_type": "stream", 632 | "text": [ 633 | "INFO:llama_index.vector_stores.redis:Using filters: *\n", 634 | "Using filters: *\n", 635 | "INFO:llama_index.vector_stores.redis:Querying index chevy_docs\n", 636 | "Querying index chevy_docs\n", 637 | "INFO:llama_index.vector_stores.redis:Found 2 results for query with id ['blog/vector_a8ee72a5-fbef-4cea-acc7-633809a5777c', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n", 638 | "Found 2 results for query with id ['blog/vector_a8ee72a5-fbef-4cea-acc7-633809a5777c', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n", 639 | "\n", 640 | " The main differences between the three engine types available for the Chevy Colorado are the power\n", 641 | "output, torque output, displacement, bore and stroke, compression ratio, block and cylinder head\n", 642 | "material, valvetrain, fuel delivery, maximum payload rating, maximum trailering weight rating, and\n", 643 | "estimated fuel economy. The 2.5L DOHC I-4 engine has 200 hp @ 6300 rpm and 191 lb.-ft. of torque @\n", 644 | "4400 rpm with a displacement of 2460 cc (150 cu. in.), a bore and stroke of 88 mm x 101 mm (3.46 in.\n", 645 | "x 3.97 in.), a compression ratio of 11.3:1, a cast-aluminum block and cylinder head, dual-overhead\n", 646 | "camshafts with 4 valves per cylinder and Variable Valve Timing (VVT), direct injection, a maximum\n", 647 | "payload rating of 1,420 lbs. and a maximum trailering weight rating of 3,500 lbs., with an EPA-\n", 648 | "estimated fuel economy of 19 MPG city/25 highway for 2WD and 19 MPG city/24 highway for 4x4. The\n", 649 | "Chevy Colorado can also be equipped with an AEV rear bumper with recovery points, five AEV hot-\n", 650 | "stamped boron steel skid plates, front and rear\n" 651 | ] 652 | } 653 | ], 654 | "source": [ 655 | "response = query_engine.query(\"What are the main differences between the three engine types available for the Chevy Colorado?\")\n", 656 | "print(\"\\n\", textwrap.fill(str(response), 100))" 657 | ] 658 | } 659 | ], 660 | "metadata": { 661 | "kernelspec": { 662 | "display_name": "Python 3 (ipykernel)", 663 | "language": "python", 664 | "name": "python3" 665 | }, 666 | "language_info": { 667 | "codemirror_mode": { 668 | "name": "ipython", 669 | "version": 3 670 | }, 671 | "file_extension": ".py", 672 | "mimetype": "text/x-python", 673 | "name": "python", 674 | "nbconvert_exporter": "python", 675 | "pygments_lexer": "ipython3", 676 | "version": "3.11.6" 677 | } 678 | }, 679 | "nbformat": 4, 680 | "nbformat_minor": 5 681 | } 682 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pypdf 2 | PyPDF2 3 | transformers 4 | tiktoken 5 | ipywidgets 6 | ipywidgets 7 | nltk 8 | redis>=5.0.0 9 | llama_index==0.8.58 10 | openai==0.28.1 11 | --------------------------------------------------------------------------------