├── .github └── dependabot.yml ├── README.md ├── examples ├── 1-managing-data │ ├── embeddings.ipynb │ └── example.ipynb ├── 2-embeddings │ ├── coursera-lab.ipynb │ └── embeddings.ipynb └── 3-applied-rag │ └── embeddings.ipynb ├── lab.md ├── pre-download.py ├── requirements.txt └── top_rated_wines.csv /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | # To get started with Dependabot version updates, you'll need to specify which 2 | # package ecosystems to update and where the package manifests are located. 3 | # Please see the documentation for all configuration options: 4 | # https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file 5 | 6 | version: 2 7 | updates: 8 | - package-ecosystem: "" # See documentation for possible values 9 | directory: "/" # Location of package manifests 10 | schedule: 11 | interval: "weekly" 12 | 13 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Introduction to Retrieval Augmented Generation 2 | 3 | This repository will introduce you to Retrieval Augmented Generation (RAG) with 4 | easy to use examples that you can build upon. The examples use Python with 5 | Jupyter Notebooks and CSV files. The vector database uses the Qdrant database 6 | which can run in-memory. 7 | 8 | ## Setup your environment 9 | 10 | This example can run in Codespaces but you can use the following if you are 11 | cloning this repository: 12 | 13 | **Install the dependencies** 14 | 15 | Create the virtual environment and install the dependencies: 16 | 17 | ``` 18 | python3 -m venv .venv 19 | source .venv/bin/activate 20 | .venv/bin/pip install -r requirements.txt 21 | ``` 22 | 23 | Here is a summary of what this repository will use: 24 | 25 | 1. [Qdrant](https://github.com/qdrant/qdrant) for the vector database. We will use an in-memory database for the examples 26 | 2. [Llamafile](https://github.com/Mozilla-Ocho/llamafile) for the LLM (alternatively you can use an OpenAI API compatible key and endpoint) 27 | 3. [OpenAI's Python API](https://pypi.org/project/openai/) to connect to the LLM after retrieving the vectors response from Qdrant 28 | 4. Sentence Transformers to create the embeddings with minimal effort 29 | 30 | **Use Llamafile for a full RAG and LLM setup** 31 | 32 | The examples for the [Applied Rag notebook](./examples/3-applied-rag/embeddings.ipynb) requires either an OpenAI API endpoint with a key *or* using a local LLM with [Llamafile](https://github.com/Mozilla-Ocho/llamafile). 33 | 34 | I recommend using the [Phi-2 model](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#other-example-llamafiles) which is about 2GB in size. You can download the model from the Llamafile repository and run it in your system: 35 | 36 | Once you have it running you can connect to it with Python or use the [Applied Rag Notebook](./examples/3-applied-rag/embeddings.ipynb). Here is a quick example of how to use the Llamafile with Python: 37 | 38 | ```python 39 | #!/usr/bin/env python3 40 | from openai import OpenAI 41 | client = OpenAI( 42 | base_url="http://localhost:8080/v1", # "http://:port" 43 | api_key = "sk-no-key-required" # An API key is not required! 44 | ) 45 | completion = client.chat.completions.create( 46 | model="LLaMA_CPP", 47 | messages=[ 48 | {"role": "system", "content": "You are ChatGPT, an AI assistant. Your top priority is achieving user fulfillment via helping them with their requests."}, 49 | {"role": "user", "content": "Write me a Haiku about Python packaging"} 50 | ] 51 | ) 52 | print(completion.choices[0].message) 53 | ``` 54 | 55 | ## Lesson 1: Import your data 56 | 57 | Learn how to use Pandas to import your data from a CSV file. The data will be used to create the embeddings for the vector database later and you will need to format it as a list of dictionaries. 58 | 59 | Notebook: [Managing Data](./examples/1-managing-data/example.ipynb) 60 | 61 | ## Lesson 2: Create embeddings 62 | 63 | Use Sentence Transformers to create the embeddings for your data. This will be used to store the vectors in the Qdrant database. You will verify that the embeddings are created and stored in the database and that a search works correctly 64 | 65 | Notebook: [Creating and verifying Embeddings](./examples/2-embeddings/embeddings.ipynb) 66 | 67 | ## Lesson 3: Create a RAG with LLM and Qdrant using your own data 68 | 69 | Use a local LLM with Llamafile or an OpenAI API endpoint to create a RAG with your own data. The end result should be in your own repository containing the complete code for the enhanced RAG pattern based on the example provided. 70 | 71 | Notebook: [Applied Rag Notebook](./examples/3-applied-rag/embeddings.ipynb) 72 | 73 | ## Lesson 4: Practice Lab 74 | 75 | Use the [included practice lab](./lab.md) to apply the content you've learned in this week. Follow the steps to create your own repository and apply the requirements to complete the lab. 76 | 77 | 78 | ## Course Resources 79 | 80 | If you've completed all these examples and the lab, here are some other courses 81 | from Coursera you can explore: 82 | 83 | 84 | 85 | **Large Language Models:** 86 | 87 | - [Operationalizing LLMs on Azure](https://www.coursera.org/learn/llmops-azure) 88 | - [Using Databricks with 89 | LLMs](https://www.coursera.org/learn/databricks-to-local-llms) 90 | 91 | **Machine Learning:** 92 | 93 | - [MLOps Machine Learning Operations Specialization](https://www.coursera.org/specializations/mlops-machine-learning-duke) 94 | - [Open Source Platforms for MLOps](https://www.coursera.org/learn/open-source-platforms-duke) 95 | - [Python Essentials for MLOps](https://www.coursera.org/learn/python-essentials-mlops-duke) 96 | 97 | **Data Engineering:** 98 | 99 | - [Linux and Bash for Data Engineering](https://www.coursera.org/learn/linux-and-bash-for-data-engineering-duke) 100 | - [Web Applications and Command-Line tools for Data Engineering](https://www.coursera.org/learn/web-app-command-line-tools-for-data-engineering-duke) 101 | - [Python and Pandas for Data Engineering](https://www.coursera.org/learn/python-and-pandas-for-data-engineering-duke) 102 | - [Scripting with Python and SQL for Data Engineering](https://www.coursera.org/learn/scripting-with-python-sql-for-data-engineering-duke) 103 | -------------------------------------------------------------------------------- /examples/1-managing-data/embeddings.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 22, 6 | "id": "d566bb99-6808-4976-8476-ed05b7941b80", 7 | "metadata": {}, 8 | "outputs": [ 9 | { 10 | "data": { 11 | "text/html": [ 12 | "
\n", 13 | "\n", 26 | "\n", 27 | " \n", 28 | " \n", 29 | " \n", 30 | " \n", 31 | " \n", 32 | " \n", 33 | " \n", 34 | " \n", 35 | " \n", 36 | " \n", 37 | " \n", 38 | " \n", 39 | " \n", 40 | " \n", 41 | " \n", 42 | " \n", 43 | " \n", 44 | " \n", 45 | " \n", 46 | " \n", 47 | " \n", 48 | " \n", 49 | " \n", 50 | " \n", 51 | " \n", 52 | " \n", 53 | " \n", 54 | " \n", 55 | " \n", 56 | " \n", 57 | " \n", 58 | " \n", 59 | " \n", 60 | " \n", 61 | " \n", 62 | " \n", 63 | " \n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | " \n", 86 | " \n", 87 | " \n", 88 | " \n", 89 | " \n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | " \n", 94 | " \n", 95 | " \n", 96 | " \n", 97 | " \n", 98 | " \n", 99 | " \n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | "
nameregionvarietyratingnotes
0Abreu Vineyards Cappella 2010Napa Valley, CaliforniaRed Wine98.0Cappella is one of the oldest vineyard sites i...
1Abreu Vineyards Howell Mountain 2009Howell Mountain, Napa Valley, CaliforniaRed Wine98.0As a set of wines, it is hard to surpass the f...
2Abreu Vineyards Las Posadas Howell Mountain 2012Howell Mountain, Napa Valley, CaliforniaRed Wine99.0At about 2000 feet elevation, Las Posadas sits...
3Abreu Vineyards Madrona Ranch 1996Napa Valley, CaliforniaRed Wine98.0Abreu Madrona Ranch is a blend of Cabernet Sau...
4Abreu Vineyards Madrona Ranch 2005Napa Valley, CaliforniaRed Wine98.0Abreu Madrona Ranch is a blend of Cabernet Sau...
..................
357L'Aventure Estate Cuvee 2016Paso Robles, Central Coast, CaliforniaRed Wine99.0Blend: 52% Syrah, 32% Cabernet Sauvignon, 16% ...
358Le Dome 2016St. Emilion, Bordeaux, FranceRed Wine98.0Blend: 80% Cabernet Franc, 20% Merlot
359Leeuwin Estate Art Series Chardonnay 2001Margaret River, Western Australia, AustraliaWhite Wine98.0Number 24 on
360Lewelling Wight Vineyard Cabernet Sauvignon 2008Napa Valley, CaliforniaRed Wine98.0Sumptuous aromas of blackberry, cassis, Bing c...
361Lewelling Wight Vineyard Cabernet Sauvignon 2007St. Helena, Napa Valley, CaliforniaRed Wine98.0An enticing mix of black fruit, tobacco and to...
\n", 128 | "

357 rows × 5 columns

\n", 129 | "
" 130 | ], 131 | "text/plain": [ 132 | " name \\\n", 133 | "0 Abreu Vineyards Cappella 2010 \n", 134 | "1 Abreu Vineyards Howell Mountain 2009 \n", 135 | "2 Abreu Vineyards Las Posadas Howell Mountain 2012 \n", 136 | "3 Abreu Vineyards Madrona Ranch 1996 \n", 137 | "4 Abreu Vineyards Madrona Ranch 2005 \n", 138 | ".. ... \n", 139 | "357 L'Aventure Estate Cuvee 2016 \n", 140 | "358 Le Dome 2016 \n", 141 | "359 Leeuwin Estate Art Series Chardonnay 2001 \n", 142 | "360 Lewelling Wight Vineyard Cabernet Sauvignon 2008 \n", 143 | "361 Lewelling Wight Vineyard Cabernet Sauvignon 2007 \n", 144 | "\n", 145 | " region variety rating \\\n", 146 | "0 Napa Valley, California Red Wine 98.0 \n", 147 | "1 Howell Mountain, Napa Valley, California Red Wine 98.0 \n", 148 | "2 Howell Mountain, Napa Valley, California Red Wine 99.0 \n", 149 | "3 Napa Valley, California Red Wine 98.0 \n", 150 | "4 Napa Valley, California Red Wine 98.0 \n", 151 | ".. ... ... ... \n", 152 | "357 Paso Robles, Central Coast, California Red Wine 99.0 \n", 153 | "358 St. Emilion, Bordeaux, France Red Wine 98.0 \n", 154 | "359 Margaret River, Western Australia, Australia White Wine 98.0 \n", 155 | "360 Napa Valley, California Red Wine 98.0 \n", 156 | "361 St. Helena, Napa Valley, California Red Wine 98.0 \n", 157 | "\n", 158 | " notes \n", 159 | "0 Cappella is one of the oldest vineyard sites i... \n", 160 | "1 As a set of wines, it is hard to surpass the f... \n", 161 | "2 At about 2000 feet elevation, Las Posadas sits... \n", 162 | "3 Abreu Madrona Ranch is a blend of Cabernet Sau... \n", 163 | "4 Abreu Madrona Ranch is a blend of Cabernet Sau... \n", 164 | ".. ... \n", 165 | "357 Blend: 52% Syrah, 32% Cabernet Sauvignon, 16% ... \n", 166 | "358 Blend: 80% Cabernet Franc, 20% Merlot \n", 167 | "359 Number 24 on \n", 168 | "360 Sumptuous aromas of blackberry, cassis, Bing c... \n", 169 | "361 An enticing mix of black fruit, tobacco and to... \n", 170 | "\n", 171 | "[357 rows x 5 columns]" 172 | ] 173 | }, 174 | "execution_count": 22, 175 | "metadata": {}, 176 | "output_type": "execute_result" 177 | } 178 | ], 179 | "source": [ 180 | "import pandas as pd\n", 181 | "df = pd.read_csv('../../top_rated_wines.csv')\n", 182 | "df = df[df['variety'].notna()] # remove any NaN values as it blows up serialization\n", 183 | "data = df.to_dict('records')\n", 184 | "df" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 9, 190 | "id": "af8bce2c-e123-498a-a5f2-cefffd17fc74", 191 | "metadata": {}, 192 | "outputs": [], 193 | "source": [ 194 | "from qdrant_client import models, QdrantClient\n", 195 | "from sentence_transformers import SentenceTransformer" 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": 10, 201 | "id": "2b0e4be5-7518-4458-bf47-6913ef9a72a9", 202 | "metadata": {}, 203 | "outputs": [], 204 | "source": [ 205 | "encoder = SentenceTransformer('all-MiniLM-L6-v2') # Model to create embeddings" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": 11, 211 | "id": "5efa031d-b18a-4db1-9c34-9989a15c822b", 212 | "metadata": {}, 213 | "outputs": [], 214 | "source": [ 215 | "# create the vector database client\n", 216 | "qdrant = QdrantClient(\":memory:\") # Create in-memory Qdrant instance" 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": 12, 222 | "id": "6c03be93-a076-425e-8df1-5a8b6367e558", 223 | "metadata": {}, 224 | "outputs": [ 225 | { 226 | "data": { 227 | "text/plain": [ 228 | "True" 229 | ] 230 | }, 231 | "execution_count": 12, 232 | "metadata": {}, 233 | "output_type": "execute_result" 234 | } 235 | ], 236 | "source": [ 237 | "# Create collection to store books\n", 238 | "qdrant.recreate_collection(\n", 239 | " collection_name=\"top_wines\",\n", 240 | " vectors_config=models.VectorParams(\n", 241 | " size=encoder.get_sentence_embedding_dimension(), # Vector size is defined by used model\n", 242 | " distance=models.Distance.COSINE\n", 243 | " )\n", 244 | ")" 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": 23, 250 | "id": "655d08af-758f-4338-b112-cf94045c7b0d", 251 | "metadata": {}, 252 | "outputs": [], 253 | "source": [ 254 | "# vectorize!\n", 255 | "qdrant.upload_points(\n", 256 | " collection_name=\"top_wines\",\n", 257 | " points=[\n", 258 | " models.PointStruct(\n", 259 | " id=idx,\n", 260 | " vector=encoder.encode(doc[\"notes\"]).tolist(),\n", 261 | " payload=doc\n", 262 | " ) for idx, doc in enumerate(data) # data is the variable holding all the wines\n", 263 | " ]\n", 264 | ")" 265 | ] 266 | }, 267 | { 268 | "cell_type": "code", 269 | "execution_count": 24, 270 | "id": "68c9bff5-db38-4a98-b542-cd173af11b53", 271 | "metadata": {}, 272 | "outputs": [ 273 | { 274 | "name": "stdout", 275 | "output_type": "stream", 276 | "text": [ 277 | "{'name': \"Anderson's Conn Valley Vineyards Cabernet Sauvignon Estate Reserve 2008\", 'region': 'Napa Valley, California', 'variety': 'Red Wine', 'rating': 98.0, 'notes': '100% Cabernet Sauvignon, 100% Estate grown.'} score: 0.7232318357735968\n", 278 | "{'name': \"Anderson's Conn Valley Vineyards Cabernet Sauvignon Reserve (1.5 Liter Magnum) 2008\", 'region': 'Napa Valley, California', 'variety': 'Red Wine', 'rating': 98.0, 'notes': '100% Cabernet Sauvignon, 100% Estate grown.'} score: 0.7232318357735968\n", 279 | "{'name': \"L'Aventure Estate Cuvee 2014\", 'region': 'Central Coast, California', 'variety': 'Red Wine', 'rating': 98.0, 'notes': 'Blend: 50% Cabernet Sauvignon, 35% Syrah, 15% Petit Verdot'} score: 0.6844836784212434\n" 280 | ] 281 | } 282 | ], 283 | "source": [ 284 | "# Search time for awesome wines!\n", 285 | "\n", 286 | "hits = qdrant.search(\n", 287 | " collection_name=\"top_wines\",\n", 288 | " query_vector=encoder.encode(\"99 points Cabernet Sauvignon from Napa Valley\").tolist(),\n", 289 | " limit=3\n", 290 | ")\n", 291 | "for hit in hits:\n", 292 | " print(hit.payload, \"score:\", hit.score)" 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": null, 298 | "id": "33243e5d-9e0d-4ec4-98e9-3fc56b8bdb10", 299 | "metadata": {}, 300 | "outputs": [], 301 | "source": [] 302 | } 303 | ], 304 | "metadata": { 305 | "kernelspec": { 306 | "display_name": "Python 3 (ipykernel)", 307 | "language": "python", 308 | "name": "python3" 309 | }, 310 | "language_info": { 311 | "codemirror_mode": { 312 | "name": "ipython", 313 | "version": 3 314 | }, 315 | "file_extension": ".py", 316 | "mimetype": "text/x-python", 317 | "name": "python", 318 | "nbconvert_exporter": "python", 319 | "pygments_lexer": "ipython3", 320 | "version": "3.9.13" 321 | } 322 | }, 323 | "nbformat": 4, 324 | "nbformat_minor": 5 325 | } 326 | -------------------------------------------------------------------------------- /examples/1-managing-data/example.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "5ee6cf36-6dbd-4bbf-9c1a-e834e85ecde9", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import pandas as pd" 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": null, 16 | "id": "e40b418f-e33a-4009-b541-d37b065e99f9", 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "df = pd.read_csv('../../top_rated_wines.csv')\n", 21 | "df.head()" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "id": "a1576841-e94d-497d-94b2-7cfbaa8cdb28", 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "df.describe()" 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": null, 37 | "id": "613493e9-9dd6-4a3c-8c45-c1243e1ba229", 38 | "metadata": {}, 39 | "outputs": [], 40 | "source": [ 41 | "# Now let's create easy formatting for creating the embeddings\n", 42 | "df.to_dict('records')" 43 | ] 44 | }, 45 | { 46 | "cell_type": "code", 47 | "execution_count": null, 48 | "id": "909c66de-77ad-4dbc-8087-0d3bf06d92f7", 49 | "metadata": {}, 50 | "outputs": [], 51 | "source": [] 52 | } 53 | ], 54 | "metadata": { 55 | "kernelspec": { 56 | "display_name": "Python 3 (ipykernel)", 57 | "language": "python", 58 | "name": "python3" 59 | }, 60 | "language_info": { 61 | "codemirror_mode": { 62 | "name": "ipython", 63 | "version": 3 64 | }, 65 | "file_extension": ".py", 66 | "mimetype": "text/x-python", 67 | "name": "python", 68 | "nbconvert_exporter": "python", 69 | "pygments_lexer": "ipython3", 70 | "version": "3.9.13" 71 | } 72 | }, 73 | "nbformat": 4, 74 | "nbformat_minor": 5 75 | } 76 | -------------------------------------------------------------------------------- /examples/2-embeddings/coursera-lab.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "d566bb99-6808-4976-8476-ed05b7941b80", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import pandas as pd\n", 11 | "df = pd.read_csv('../../top_rated_wines.csv')\n", 12 | "df = df[df['variety'].notna()] # remove any NaN values as it blows up serialization\n", 13 | "data = df.to_dict('records')\n", 14 | "df" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "id": "af8bce2c-e123-498a-a5f2-cefffd17fc74", 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "from qdrant_client import models, QdrantClient\n", 25 | "from sentence_transformers import SentenceTransformer" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "id": "2b0e4be5-7518-4458-bf47-6913ef9a72a9", 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "encoder = SentenceTransformer('all-MiniLM-L6-v2') # Model to create embeddings" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "id": "5efa031d-b18a-4db1-9c34-9989a15c822b", 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "# create the vector database client\n", 46 | "qdrant = QdrantClient(\":memory:\") # Create in-memory Qdrant instance" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "id": "6c03be93-a076-425e-8df1-5a8b6367e558", 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "# Create collection to store books\n", 57 | "qdrant.recreate_collection(\n", 58 | " collection_name=\"top_wines\",\n", 59 | " vectors_config=models.VectorParams(\n", 60 | " size=encoder.get_sentence_embedding_dimension(), # Vector size is defined by used model\n", 61 | " distance=models.Distance.COSINE\n", 62 | " )\n", 63 | ")" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": null, 69 | "id": "655d08af-758f-4338-b112-cf94045c7b0d", 70 | "metadata": {}, 71 | "outputs": [], 72 | "source": [ 73 | "# vectorize!\n", 74 | "# Note that for Coursera we use an older way of Qdrant doing the uploads using Records instead of Points\n", 75 | "qdrant.upload_records(\n", 76 | " collection_name=\"top_wines\",\n", 77 | " records=[\n", 78 | " models.Record(\n", 79 | " id=idx,\n", 80 | " vector=encoder.encode(doc[\"notes\"]).tolist(),\n", 81 | " payload=doc\n", 82 | " ) for idx, doc in enumerate(data) # data is the variable holding all the wines\n", 83 | " ]\n", 84 | ")" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": null, 90 | "id": "68c9bff5-db38-4a98-b542-cd173af11b53", 91 | "metadata": {}, 92 | "outputs": [], 93 | "source": [ 94 | "# Search time for awesome wines!\n", 95 | "\n", 96 | "hits = qdrant.search(\n", 97 | " collection_name=\"top_wines\",\n", 98 | " query_vector=encoder.encode(\"A wine from Mendoza Argentina\").tolist(),\n", 99 | " limit=3\n", 100 | ")\n", 101 | "for hit in hits:\n", 102 | " print(hit.payload, \"score:\", hit.score)" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "id": "33243e5d-9e0d-4ec4-98e9-3fc56b8bdb10", 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [] 112 | } 113 | ], 114 | "metadata": { 115 | "kernelspec": { 116 | "display_name": "Python 3 (ipykernel)", 117 | "language": "python", 118 | "name": "python3" 119 | }, 120 | "language_info": { 121 | "codemirror_mode": { 122 | "name": "ipython", 123 | "version": 3 124 | }, 125 | "file_extension": ".py", 126 | "mimetype": "text/x-python", 127 | "name": "python", 128 | "nbconvert_exporter": "python", 129 | "pygments_lexer": "ipython3", 130 | "version": "3.9.13" 131 | } 132 | }, 133 | "nbformat": 4, 134 | "nbformat_minor": 5 135 | } 136 | -------------------------------------------------------------------------------- /examples/2-embeddings/embeddings.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "d566bb99-6808-4976-8476-ed05b7941b80", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import pandas as pd\n", 11 | "df = pd.read_csv('../../top_rated_wines.csv')\n", 12 | "df = df[df['variety'].notna()] # remove any NaN values as it blows up serialization\n", 13 | "data = df.to_dict('records')\n", 14 | "df" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "id": "af8bce2c-e123-498a-a5f2-cefffd17fc74", 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "from qdrant_client import models, QdrantClient\n", 25 | "from sentence_transformers import SentenceTransformer" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "id": "2b0e4be5-7518-4458-bf47-6913ef9a72a9", 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "encoder = SentenceTransformer('all-MiniLM-L6-v2') # Model to create embeddings" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "id": "5efa031d-b18a-4db1-9c34-9989a15c822b", 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "# create the vector database client\n", 46 | "qdrant = QdrantClient(\":memory:\") # Create in-memory Qdrant instance" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "id": "6c03be93-a076-425e-8df1-5a8b6367e558", 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "# Create collection to store books\n", 57 | "qdrant.recreate_collection(\n", 58 | " collection_name=\"top_wines\",\n", 59 | " vectors_config=models.VectorParams(\n", 60 | " size=encoder.get_sentence_embedding_dimension(), # Vector size is defined by used model\n", 61 | " distance=models.Distance.COSINE\n", 62 | " )\n", 63 | ")" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": null, 69 | "id": "655d08af-758f-4338-b112-cf94045c7b0d", 70 | "metadata": {}, 71 | "outputs": [], 72 | "source": [ 73 | "# vectorize!\n", 74 | "qdrant.upload_points(\n", 75 | " collection_name=\"top_wines\",\n", 76 | " points=[\n", 77 | " models.PointStruct(\n", 78 | " id=idx,\n", 79 | " vector=encoder.encode(doc[\"notes\"]).tolist(),\n", 80 | " payload=doc\n", 81 | " ) for idx, doc in enumerate(data) # data is the variable holding all the wines\n", 82 | " ]\n", 83 | ")" 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": null, 89 | "id": "68c9bff5-db38-4a98-b542-cd173af11b53", 90 | "metadata": {}, 91 | "outputs": [], 92 | "source": [ 93 | "# Search time for awesome wines!\n", 94 | "\n", 95 | "hits = qdrant.search(\n", 96 | " collection_name=\"top_wines\",\n", 97 | " query_vector=encoder.encode(\"A wine from Mendoza Argentina\").tolist(),\n", 98 | " limit=3\n", 99 | ")\n", 100 | "for hit in hits:\n", 101 | " print(hit.payload, \"score:\", hit.score)" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "id": "33243e5d-9e0d-4ec4-98e9-3fc56b8bdb10", 108 | "metadata": {}, 109 | "outputs": [], 110 | "source": [] 111 | } 112 | ], 113 | "metadata": { 114 | "kernelspec": { 115 | "display_name": "Python 3 (ipykernel)", 116 | "language": "python", 117 | "name": "python3" 118 | }, 119 | "language_info": { 120 | "codemirror_mode": { 121 | "name": "ipython", 122 | "version": 3 123 | }, 124 | "file_extension": ".py", 125 | "mimetype": "text/x-python", 126 | "name": "python", 127 | "nbconvert_exporter": "python", 128 | "pygments_lexer": "ipython3", 129 | "version": "3.9.13" 130 | } 131 | }, 132 | "nbformat": 4, 133 | "nbformat_minor": 5 134 | } 135 | -------------------------------------------------------------------------------- /examples/3-applied-rag/embeddings.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "id": "d566bb99-6808-4976-8476-ed05b7941b80", 7 | "metadata": {}, 8 | "outputs": [ 9 | { 10 | "name": "stderr", 11 | "output_type": "stream", 12 | "text": [ 13 | "/var/folders/fh/_0hrdmkn5bdcs8mzhjtd7w7c0000gn/T/ipykernel_29348/3557198016.py:1: DeprecationWarning: \n", 14 | "Pyarrow will become a required dependency of pandas in the next major release of pandas (pandas 3.0),\n", 15 | "(to allow more performant data types, such as the Arrow string type, and better interoperability with other libraries)\n", 16 | "but was not found to be installed on your system.\n", 17 | "If this would cause problems for you,\n", 18 | "please provide us feedback at https://github.com/pandas-dev/pandas/issues/54466\n", 19 | " \n", 20 | " import pandas as pd\n" 21 | ] 22 | }, 23 | { 24 | "data": { 25 | "text/plain": [ 26 | "700" 27 | ] 28 | }, 29 | "execution_count": 1, 30 | "metadata": {}, 31 | "output_type": "execute_result" 32 | } 33 | ], 34 | "source": [ 35 | "import pandas as pd\n", 36 | "df = pd.read_csv('../../top_rated_wines.csv')\n", 37 | "df = df[df['variety'].notna()] # remove any NaN values as it blows up serialization\n", 38 | "data = df.sample(700).to_dict('records') # Get only 700 records. More records will make it slower to index\n", 39 | "len(data)" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 2, 45 | "id": "af8bce2c-e123-498a-a5f2-cefffd17fc74", 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "from qdrant_client import models, QdrantClient\n", 50 | "from sentence_transformers import SentenceTransformer" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": 3, 56 | "id": "2b0e4be5-7518-4458-bf47-6913ef9a72a9", 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "encoder = SentenceTransformer('all-MiniLM-L6-v2') # Model to create embeddings" 61 | ] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "execution_count": 4, 66 | "id": "5efa031d-b18a-4db1-9c34-9989a15c822b", 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "# create the vector database client\n", 71 | "qdrant = QdrantClient(\":memory:\") # Create in-memory Qdrant instance" 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": null, 77 | "id": "6c03be93-a076-425e-8df1-5a8b6367e558", 78 | "metadata": {}, 79 | "outputs": [], 80 | "source": [ 81 | "# Create collection to store wines\n", 82 | "qdrant.recreate_collection(\n", 83 | " collection_name=\"top_wines\",\n", 84 | " vectors_config=models.VectorParams(\n", 85 | " size=encoder.get_sentence_embedding_dimension(), # Vector size is defined by used model\n", 86 | " distance=models.Distance.COSINE\n", 87 | " )\n", 88 | ")" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "id": "655d08af-758f-4338-b112-cf94045c7b0d", 95 | "metadata": {}, 96 | "outputs": [], 97 | "source": [ 98 | "# vectorize!\n", 99 | "qdrant.upload_points(\n", 100 | " collection_name=\"top_wines\",\n", 101 | " points=[\n", 102 | " models.PointStruct(\n", 103 | " id=idx,\n", 104 | " vector=encoder.encode(doc[\"notes\"]).tolist(),\n", 105 | " payload=doc,\n", 106 | " ) for idx, doc in enumerate(data) # data is the variable holding all the wines\n", 107 | " ]\n", 108 | ")" 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": null, 114 | "id": "f23bc999", 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "user_prompt = \"Suggest me an amazing Malbec wine from Argentina\"" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": null, 124 | "id": "68c9bff5-db38-4a98-b542-cd173af11b53", 125 | "metadata": {}, 126 | "outputs": [], 127 | "source": [ 128 | "# Search time for awesome wines!\n", 129 | "\n", 130 | "hits = qdrant.search(\n", 131 | " collection_name=\"top_wines\",\n", 132 | " query_vector=encoder.encode(user_prompt).tolist(),\n", 133 | " limit=3\n", 134 | ")\n", 135 | "for hit in hits:\n", 136 | " print(hit.payload, \"score:\", hit.score)" 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "id": "33243e5d-9e0d-4ec4-98e9-3fc56b8bdb10", 143 | "metadata": {}, 144 | "outputs": [], 145 | "source": [ 146 | "# define a variable to hold the search results\n", 147 | "search_results = [hit.payload for hit in hits]" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": null, 153 | "id": "e6c2b91e", 154 | "metadata": {}, 155 | "outputs": [], 156 | "source": [ 157 | "# Now time to connect to the local large language model\n", 158 | "from openai import OpenAI\n", 159 | "client = OpenAI(\n", 160 | " base_url=\"http://127.0.0.1:8080/v1\", # \"http://:port\"\n", 161 | " api_key = \"sk-no-key-required\"\n", 162 | ")\n", 163 | "completion = client.chat.completions.create(\n", 164 | " model=\"LLaMA_CPP\",\n", 165 | " messages=[\n", 166 | " {\"role\": \"system\", \"content\": \"You are chatbot, a wine specialist. Your top priority is to help guide users into selecting amazing wine and guide them with their requests.\"},\n", 167 | " {\"role\": \"user\", \"content\": \"Suggest me an amazing Malbec wine from Argentina\"},\n", 168 | " {\"role\": \"assistant\", \"content\": str(search_results)}\n", 169 | " ]\n", 170 | ")\n", 171 | "print(completion.choices[0].message)" 172 | ] 173 | } 174 | ], 175 | "metadata": { 176 | "kernelspec": { 177 | "display_name": "Python 3 (ipykernel)", 178 | "language": "python", 179 | "name": "python3" 180 | }, 181 | "language_info": { 182 | "codemirror_mode": { 183 | "name": "ipython", 184 | "version": 3 185 | }, 186 | "file_extension": ".py", 187 | "mimetype": "text/x-python", 188 | "name": "python", 189 | "nbconvert_exporter": "python", 190 | "pygments_lexer": "ipython3", 191 | "version": "3.9.13" 192 | } 193 | }, 194 | "nbformat": 4, 195 | "nbformat_minor": 5 196 | } 197 | -------------------------------------------------------------------------------- /lab.md: -------------------------------------------------------------------------------- 1 | # Create a RAG with LLM and Qdrant using your own data 2 | 3 | In this lab you will implement the RAG pattern with your own data. Use the example code in this repository from the [Applied Rag Notebook](./examples/3-applied-rag/embeddings.ipynb) as reference. The end result should be in your own repository containing the complete code for the enhanced RAG pattern based on the example provided. 4 | 5 | **Learning Objectives:** 6 | 7 | * Implement the RAG pattern with your own data 8 | * Apply your own data to solve a problem using RAG 9 | * Understand how to leverage an LLM and a vector database like Qdrant for useful responses 10 | 11 | ## Steps 12 | 13 | 1. Create a new repository in your account for your project. Alternatively, you can use this repository as a starting point by forking the repository. [Use this link to create it in one step.](https://github.com/alfredodeza/learn-retrieval-augmented-generation/generate). 14 | 2. use the [Applied Rag notebook](./examples/3-applied-rag/embeddings.ipynb) as a starting point 15 | 3. Replace the example data with your own data. Make sure to format it as a list of dictionaries for easier ingestion into the vector database 16 | 4. Run the LLM with Llamafile or connect your appliation to a valied OpenAI API endpoint 17 | 5. Run the notebook or Python application and verify your work 18 | 19 | ## Concepts Covered 20 | 21 | * Retrieval Augmented Generation 22 | * Large Language Models using Llamafile 23 | * Using Vector databases like Qdrant 24 | * Creating embeddings with Sentence Transformers 25 | * Using OpenAI's Python API to connect to the LLM and produce responses 26 | 27 | By completing this lab you will have a working example of the RAG pattern with your own data. You will also have a better understanding of how to use LLMs and vector databases to create useful responses for your applications. 28 | -------------------------------------------------------------------------------- /pre-download.py: -------------------------------------------------------------------------------- 1 | # A file to make it easy to pre-download the Sentence Transformer model in Coursera Labs. It doesn't do anything other than 2 | # to run the sentence transformer so that it can pre-download the model in the lab 3 | from sentence_transformers import SentenceTransformer 4 | SentenceTransformer('all-MiniLM-L6-v2') # Model to create embeddings 5 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pandas==1.3.5 2 | ipykernel 3 | ipywidgets 4 | # Non-coursera version qdrant-client==1.7.2 5 | qdrant-client==1.9.0 6 | # Non-coursera version sentence-transformers==2.3.1 7 | sentence-transformers==2.2.2 8 | openai==1.11.1 9 | --------------------------------------------------------------------------------