├── 1_1-First_Chatbot_OpenAI.ipynb
├── 1_2-Easy_NL2SQL.ipynb
├── 1_3-Intro_Prompt_Engineering.ipynb
├── 2_1_Vector_Databases_LLMs.ipynb
├── 2_2-ChromaDB Sever mode.ipynb
├── 2_3-ChromaDB Client.ipynb
├── 3_1_RAG_langchain.ipynb
├── 3_2_GPT_Moderation_System.ipynb
├── 3_2_LLAMA2_Moderation_Chat.ipynb
├── 3_2_OpenAI_Moderation_Chat.ipynb
├── 3_3_Data_Analyst_Agent.ipynb
├── 3_4_Medical_Assistant_Agent.ipynb
├── 4_1_bleu_evaluation.ipynb
├── 4_1_rouge_evaluations.ipynb
├── 4_2_Evaluating_summaries_embeddings.ipynb
├── 4_2_tracing_medical_agent.ipynb
├── 4_3_evaluating_rag_giskard.ipynb
├── 5_2_LoRA_Tuning.ipynb
├── 5_3_QLoRA_Tuning.ipynb
├── 5_4_Prompt_Tuning.ipynb
├── 6_1_nl2sql_prompt_OpenAI.ipynb
├── 6_2_Azure_NL2SQL_Client.ipynb
├── 6_3_AWS_Bedrock_NL2SQL_Client.ipynb
├── 6_4_nl2sql_Ollama.ipynb
├── 7_2_Aligning_DPO_phi3.ipynb
├── 979-8-8688-0514-1.jpg
├── Contributing.md
├── LICENSE.txt
└── README.md


/1_3-Intro_Prompt_Engineering.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "cells": [
  3 |     {
  4 |       "cell_type": "markdown",
  5 |       "id": "9c4700eb-00ef-433f-b574-2fb7f644226d",
  6 |       "metadata": {
  7 |         "id": "9c4700eb-00ef-433f-b574-2fb7f644226d"
  8 |       },
  9 |       "source": [
 10 |         "<div>\n",
 11 |         "<h1>Large Language Models Projects</h1>\n",
 12 |         "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
 13 |         "    <h2>Influencing the model's response with in-context learning</h2>\n",
 14 |         "   \n",
 15 |         "</div>\n",
 16 |         "\n",
 17 |         "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
 18 |         "<hr>\n",
 19 |         "\n",
 20 |         "Models: gpt-3.5-turbo\n",
 21 |         "\n",
 22 |         "Colab Environment: CPU\n",
 23 |         "\n",
 24 |         "Keys:\n",
 25 |         "* In context learning.\n",
 26 |         "* Response formating.\n",
 27 |         "* Sentiment classification.\n",
 28 |         "_______________________________\n",
 29 |         "\n",
 30 |         "\n",
 31 |         "\n"
 32 |       ]
 33 |     },
 34 |     {
 35 |       "cell_type": "markdown",
 36 |       "id": "34723a72-1601-4685-a0ba-bff544425d48",
 37 |       "metadata": {
 38 |         "id": "34723a72-1601-4685-a0ba-bff544425d48"
 39 |       },
 40 |       "source": [
 41 |         "In this notebook, we'll explore small prompt engineering techniques and recommendations that will help us elicit responses from the models that are better suited to our needs."
 42 |       ]
 43 |     },
 44 |     {
 45 |       "cell_type": "code",
 46 |       "execution_count": 1,
 47 |       "id": "fe5f658a-708f-406c-a1cb-95e3482ae760",
 48 |       "metadata": {
 49 |         "id": "fe5f658a-708f-406c-a1cb-95e3482ae760",
 50 |         "colab": {
 51 |           "base_uri": "https://localhost:8080/"
 52 |         },
 53 |         "outputId": "572e089f-049b-42a6-a660-a5efefe0c521"
 54 |       },
 55 |       "outputs": [
 56 |         {
 57 |           "output_type": "stream",
 58 |           "name": "stdout",
 59 |           "text": [
 60 |             "\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/217.8 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m \u001b[32m215.0/217.8 kB\u001b[0m \u001b[31m9.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m217.8/217.8 kB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 61 |             "\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/75.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 62 |             "\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/77.9 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 63 |             "\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/58.3 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 64 |             "\u001b[?25h"
 65 |           ]
 66 |         }
 67 |       ],
 68 |       "source": [
 69 |         "!pip install -q openai==1.1.1"
 70 |       ]
 71 |     },
 72 |     {
 73 |       "cell_type": "code",
 74 |       "execution_count": 17,
 75 |       "id": "fba193cc-d8a0-4ad2-8177-380204426859",
 76 |       "metadata": {
 77 |         "id": "fba193cc-d8a0-4ad2-8177-380204426859"
 78 |       },
 79 |       "outputs": [],
 80 |       "source": [
 81 |         "#if you need a API Key from OpenAI\n",
 82 |         "#https://platform.openai.com/account/api-keys\n",
 83 |         "\n",
 84 |         "import openai\n",
 85 |         "openai.api_key=\"your-openai-key\"\n",
 86 |         "model = \"gpt-3.5-turbo\"\n",
 87 |         "#model = \"gpt-4o\""
 88 |       ]
 89 |     },
 90 |     {
 91 |       "cell_type": "markdown",
 92 |       "id": "502cfc93-21e0-498f-9650-37bc6ddd514d",
 93 |       "metadata": {
 94 |         "id": "502cfc93-21e0-498f-9650-37bc6ddd514d"
 95 |       },
 96 |       "source": [
 97 |         "# Formating the answer with Few Shot Samples.\n",
 98 |         "\n",
 99 |         "To obtain the model's response in a specific format, we have various options, but one of the most convenient is to use Few-Shot Samples. This involves presenting the model with pairs of user queries and example responses.\n",
100 |         "\n",
101 |         "Large models like GPT-3.5 respond well to the examples provided, adapting their response to the specified format.\n",
102 |         "\n",
103 |         "Depending on the number of examples given, this technique can be referred to as:\n",
104 |         "* Zero-Shot.\n",
105 |         "* One-Shot.\n",
106 |         "* Few-Shots.\n",
107 |         "\n",
108 |         "With One Shot should be enough, and it is recommended to use a maximum of six shots. It's important to remember that this information is passed in each query and occupies space in the input prompt.\n",
109 |         "\n"
110 |       ]
111 |     },
112 |     {
113 |       "cell_type": "code",
114 |       "execution_count": 18,
115 |       "id": "a8344712-06d7-4c24-83d8-f36d62926e5e",
116 |       "metadata": {
117 |         "id": "a8344712-06d7-4c24-83d8-f36d62926e5e"
118 |       },
119 |       "outputs": [],
120 |       "source": [
121 |         "#Functio to call the model.\n",
122 |         "def return_OAIResponse(user_message, context):\n",
123 |         "\n",
124 |         "    newcontext = context.copy()\n",
125 |         "    newcontext.append({'role':'user', 'content':\"question: \" + user_message})\n",
126 |         "\n",
127 |         "    response = openai.chat.completions.create(\n",
128 |         "            model=model,\n",
129 |         "            messages=newcontext,\n",
130 |         "            temperature=1,\n",
131 |         "        )\n",
132 |         "\n",
133 |         "    return (response.choices[0].message.content)"
134 |       ]
135 |     },
136 |     {
137 |       "cell_type": "markdown",
138 |       "id": "f611d73d-9330-466d-b705-543667e1b561",
139 |       "metadata": {
140 |         "id": "f611d73d-9330-466d-b705-543667e1b561"
141 |       },
142 |       "source": [
143 |         "In this zero-shots prompt we obtain a correct response, but without formatting, as the model incorporates the information he wants."
144 |       ]
145 |     },
146 |     {
147 |       "cell_type": "code",
148 |       "execution_count": 19,
149 |       "id": "647790be-fdb8-4692-a82e-7e3a0220f72a",
150 |       "metadata": {
151 |         "colab": {
152 |           "base_uri": "https://localhost:8080/"
153 |         },
154 |         "id": "647790be-fdb8-4692-a82e-7e3a0220f72a",
155 |         "outputId": "ed07d20e-e90b-41a3-cd1a-64d84f0ac7d2"
156 |       },
157 |       "outputs": [
158 |         {
159 |           "output_type": "stream",
160 |           "name": "stdout",
161 |           "text": [
162 |             "Sebastian Vettel won the F1 2010 World Championship driving for Red Bull Racing.\n"
163 |           ]
164 |         }
165 |       ],
166 |       "source": [
167 |         "#zero-shot\n",
168 |         "context_user = [\n",
169 |         "    {'role':'system', 'content':'You are an expert in F1.'}\n",
170 |         "]\n",
171 |         "print(return_OAIResponse(\"Who won the F1 2010?\", context_user))"
172 |       ]
173 |     },
174 |     {
175 |       "cell_type": "markdown",
176 |       "id": "e87a9a0a-c1b9-4759-b52f-f6547d29b4c8",
177 |       "metadata": {
178 |         "id": "e87a9a0a-c1b9-4759-b52f-f6547d29b4c8"
179 |       },
180 |       "source": [
181 |         "Models as large and good as GPT 3.5 and GPT 4 only need a single shot to learn the output format we expect.\n"
182 |       ]
183 |     },
184 |     {
185 |       "cell_type": "code",
186 |       "execution_count": 20,
187 |       "id": "33ac7693-6cf3-44f7-b2ff-55d8a36fe775",
188 |       "metadata": {
189 |         "colab": {
190 |           "base_uri": "https://localhost:8080/"
191 |         },
192 |         "id": "33ac7693-6cf3-44f7-b2ff-55d8a36fe775",
193 |         "outputId": "9cb248e1-19e3-4ef1-f03c-d126adf4a851"
194 |       },
195 |       "outputs": [
196 |         {
197 |           "output_type": "stream",
198 |           "name": "stdout",
199 |           "text": [
200 |             "Driver: Sebastian Vettel.\n",
201 |             "Team: Red Bull Racing.\n"
202 |           ]
203 |         }
204 |       ],
205 |       "source": [
206 |         "#one-shot\n",
207 |         "context_user = [\n",
208 |         "    {'role':'system', 'content':\n",
209 |         "     \"\"\"You are an expert in F1.\n",
210 |         "\n",
211 |         "     Who won the 2000 f1 championship?\n",
212 |         "     Driver: Michael Schumacher.\n",
213 |         "     Team: Ferrari.\"\"\"}\n",
214 |         "]\n",
215 |         "print(return_OAIResponse(\"Who won the F1 2011?\", context_user))"
216 |       ]
217 |     },
218 |     {
219 |       "cell_type": "markdown",
220 |       "id": "32c454a8-181b-482b-873a-81d6ffde4674",
221 |       "metadata": {
222 |         "id": "32c454a8-181b-482b-873a-81d6ffde4674"
223 |       },
224 |       "source": [
225 |         "Smaller models, or more complicated formats, may require more than one shot. Here a sample with two shots."
226 |       ]
227 |     },
228 |     {
229 |       "cell_type": "code",
230 |       "execution_count": 21,
231 |       "id": "8ce600f7-f92e-4cf7-be4a-408f12eb39d6",
232 |       "metadata": {
233 |         "colab": {
234 |           "base_uri": "https://localhost:8080/"
235 |         },
236 |         "id": "8ce600f7-f92e-4cf7-be4a-408f12eb39d6",
237 |         "outputId": "de30bb15-f660-448b-96d4-556879da5d50"
238 |       },
239 |       "outputs": [
240 |         {
241 |           "output_type": "stream",
242 |           "name": "stdout",
243 |           "text": [
244 |             "Driver: Fernando Alonso.\n",
245 |             "Team: Renault.\n"
246 |           ]
247 |         }
248 |       ],
249 |       "source": [
250 |         "#Few shots\n",
251 |         "context_user = [\n",
252 |         "    {'role':'system', 'content':\n",
253 |         "     \"\"\"You are an expert in F1.\n",
254 |         "\n",
255 |         "     Who won the 2010 f1 championship?\n",
256 |         "     Driver: Sebastian Vettel.\n",
257 |         "     Team: Red Bull Renault.\n",
258 |         "\n",
259 |         "     Who won the 2009 f1 championship?\n",
260 |         "     Driver: Jenson Button.\n",
261 |         "     Team: BrawnGP.\"\"\"}\n",
262 |         "]\n",
263 |         "print(return_OAIResponse(\"Who won the F1 2006?\", context_user))"
264 |       ]
265 |     },
266 |     {
267 |       "cell_type": "markdown",
268 |       "id": "5f1b71c4-6583-4dcb-b987-02abf6aa4a86",
269 |       "metadata": {
270 |         "id": "5f1b71c4-6583-4dcb-b987-02abf6aa4a86"
271 |       },
272 |       "source": [
273 |         "We've been creating the prompt without using OpenAI's roles, and as we've seen, it worked correctly.\n",
274 |         "\n",
275 |         "However, the proper way to do this is by using these roles to construct the prompt, making the model's learning process even more effective.\n",
276 |         "\n",
277 |         "By not feeding it the entire prompt as if they were system commands, we enable the model to learn from a conversation, which is more realistic for it."
278 |       ]
279 |     },
280 |     {
281 |       "cell_type": "code",
282 |       "execution_count": 23,
283 |       "id": "20fa4a25-01a6-4f22-98db-ab7ccc9ba115",
284 |       "metadata": {
285 |         "colab": {
286 |           "base_uri": "https://localhost:8080/"
287 |         },
288 |         "id": "20fa4a25-01a6-4f22-98db-ab7ccc9ba115",
289 |         "outputId": "fcf7518f-1c75-43d6-9257-89913eeb2bd1"
290 |       },
291 |       "outputs": [
292 |         {
293 |           "output_type": "stream",
294 |           "name": "stdout",
295 |           "text": [
296 |             "Driver: Lewis Hamilton. \n",
297 |             "Team: Mercedes. \n",
298 |             "Points: 413.\n"
299 |           ]
300 |         }
301 |       ],
302 |       "source": [
303 |         "#Recomended solution\n",
304 |         "context_user = [\n",
305 |         "    {'role':'system', 'content':'You are and expert in f1.\\n\\n'},\n",
306 |         "    {'role':'user', 'content':'Who won the 2010 f1 championship?'},\n",
307 |         "    {'role':'assistant', 'content':\"\"\"Driver: Sebastian Vettel. \\nTeam: Red Bull. \\nPoints: 256. \"\"\"},\n",
308 |         "    {'role':'user', 'content':'Who won the 2009 f1 championship?'},\n",
309 |         "    {'role':'assistant', 'content':\"\"\"Driver: Jenson Button. \\nTeam: BrawnGP. \\nPoints: 95. \"\"\"},\n",
310 |         "]\n",
311 |         "\n",
312 |         "print(return_OAIResponse(\"Who won the F1 2019?\", context_user))"
313 |       ]
314 |     },
315 |     {
316 |       "cell_type": "markdown",
317 |       "id": "ac6f6b42-f351-496b-a7e8-1286426457eb",
318 |       "metadata": {
319 |         "id": "ac6f6b42-f351-496b-a7e8-1286426457eb"
320 |       },
321 |       "source": [
322 |         "We could also address it by using a more conventional prompt, describing what we want and how we want the format.\n",
323 |         "\n",
324 |         "However, it's essential to understand that in this case, the model is following instructions, whereas in the case of use shots, it is learning in real-time during inference."
325 |       ]
326 |     },
327 |     {
328 |       "cell_type": "code",
329 |       "execution_count": 24,
330 |       "id": "36c32a32-c348-45b2-85ee-ab4500438c49",
331 |       "metadata": {
332 |         "colab": {
333 |           "base_uri": "https://localhost:8080/"
334 |         },
335 |         "id": "36c32a32-c348-45b2-85ee-ab4500438c49",
336 |         "outputId": "c179cefd-4089-406d-d2fb-af81bb0113a0"
337 |       },
338 |       "outputs": [
339 |         {
340 |           "output_type": "stream",
341 |           "name": "stdout",
342 |           "text": [
343 |             "Drive: Lewis Hamilton\n",
344 |             "Team: Mercedes\n",
345 |             "Points: 413\n"
346 |           ]
347 |         }
348 |       ],
349 |       "source": [
350 |         "context_user = [\n",
351 |         "    {'role':'system', 'content':\"\"\"You are and expert in f1.\n",
352 |         "    You are going to answer the question of the user giving the name of the rider,\n",
353 |         "    the name of the team and the points of the champion, following the format:\n",
354 |         "    Drive:\n",
355 |         "    Team:\n",
356 |         "    Points: \"\"\"\n",
357 |         "    }\n",
358 |         "]\n",
359 |         "\n",
360 |         "print(return_OAIResponse(\"Who won the F1 2019?\", context_user))"
361 |       ]
362 |     },
363 |     {
364 |       "cell_type": "markdown",
365 |       "source": [
366 |         "Few Shots for classification.\n"
367 |       ],
368 |       "metadata": {
369 |         "id": "qZPNTLMPnkQ4"
370 |       },
371 |       "id": "qZPNTLMPnkQ4"
372 |     },
373 |     {
374 |       "cell_type": "code",
375 |       "source": [
376 |         "context_user = [\n",
377 |         "    {'role':'system', 'content':\n",
378 |         "     \"\"\"You are an expert in reviewing product opinions and classifying them as positive or negative.\n",
379 |         "\n",
380 |         "     It fulfilled its function perfectly, I think the price is fair, I would buy it again.\n",
381 |         "     Sentiment: Positive\n",
382 |         "\n",
383 |         "     It didn't work bad, but I wouldn't buy it again, maybe it's a bit expensive for what it does.\n",
384 |         "     Sentiment: Negative.\n",
385 |         "\n",
386 |         "     I wouldn't know what to say, my son uses it, but he doesn't love it.\n",
387 |         "     Sentiment: Neutral\n",
388 |         "     \"\"\"}\n",
389 |         "]\n",
390 |         "print(return_OAIResponse(\"I'm not going to return it, but I don't plan to buy it again.\", context_user))"
391 |       ],
392 |       "metadata": {
393 |         "colab": {
394 |           "base_uri": "https://localhost:8080/"
395 |         },
396 |         "id": "ejcstgTxnnX5",
397 |         "outputId": "179ce58a-8592-40dd-8e4b-282d33be9489"
398 |       },
399 |       "id": "ejcstgTxnnX5",
400 |       "execution_count": 25,
401 |       "outputs": [
402 |         {
403 |           "output_type": "stream",
404 |           "name": "stdout",
405 |           "text": [
406 |             "Sentiment: Neutral\n"
407 |           ]
408 |         }
409 |       ]
410 |     },
411 |     {
412 |       "cell_type": "code",
413 |       "source": [],
414 |       "metadata": {
415 |         "id": "ZHr_75sDqDJp"
416 |       },
417 |       "id": "ZHr_75sDqDJp",
418 |       "execution_count": null,
419 |       "outputs": []
420 |     }
421 |   ],
422 |   "metadata": {
423 |     "kernelspec": {
424 |       "display_name": "Python 3 (ipykernel)",
425 |       "language": "python",
426 |       "name": "python3"
427 |     },
428 |     "language_info": {
429 |       "codemirror_mode": {
430 |         "name": "ipython",
431 |         "version": 3
432 |       },
433 |       "file_extension": ".py",
434 |       "mimetype": "text/x-python",
435 |       "name": "python",
436 |       "nbconvert_exporter": "python",
437 |       "pygments_lexer": "ipython3",
438 |       "version": "3.10.12"
439 |     },
440 |     "colab": {
441 |       "provenance": []
442 |     }
443 |   },
444 |   "nbformat": 4,
445 |   "nbformat_minor": 5
446 | }


--------------------------------------------------------------------------------
/2_1_Vector_Databases_LLMs.ipynb:
--------------------------------------------------------------------------------
   1 | {
   2 |   "nbformat": 4,
   3 |   "nbformat_minor": 0,
   4 |   "metadata": {
   5 |     "colab": {
   6 |       "provenance": []
   7 |     },
   8 |     "kernelspec": {
   9 |       "name": "python3",
  10 |       "display_name": "Python 3"
  11 |     },
  12 |     "language_info": {
  13 |       "name": "python"
  14 |     }
  15 |   },
  16 |   "cells": [
  17 |     {
  18 |       "cell_type": "markdown",
  19 |       "source": [
  20 |         "<div>\n",
  21 |         "    <h1>Large Language Models Projects</a></h1>\n",
  22 |         "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
  23 |         "    <h2>2.1-Vector Databases with LLMs</h2>\n",
  24 |         "</div>\n",
  25 |         "\n",
  26 |         "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
  27 |         "__________\n",
  28 |         "Models: TinyLlama/TinyLlama-1.1B-Chat-v1.0\n",
  29 |         "\n",
  30 |         "Colab environment: CPU.\n",
  31 |         "\n",
  32 |         "Keys:\n",
  33 |         "* Vector Database.\n",
  34 |         "* ChromaDB.\n",
  35 |         "* RAG\n",
  36 |         "* Embeddings.\n",
  37 |         "__________\n",
  38 |         "\n",
  39 |         "\n",
  40 |         "If you are executing this notebook on Colab you will need a High RAM capacity environment, depending on the model used.\n",
  41 |         "\n",
  42 |         "If you don't have a Colab Pro acount you can execute this notebook on kaggle, since you will get more memory from the free tier.\n",
  43 |         "\n",
  44 |         "Here yo have a version of this notebook, that uses a Dolly 3B model, that can be executed on Kaggle: [Vector Databases with LLMs-Kaggle Version](https://github.com/peremartra/Large-Language-Model-Notebooks-Course/blob/main/2-Vector%20Databases%20with%20LLMs/how-to-use-a-embedding-database-with-a-llm-from-hf.ipynb)\n",
  45 |         "__________\n",
  46 |         "\n"
  47 |       ],
  48 |       "metadata": {
  49 |         "id": "8vPA--nKMhoQ"
  50 |       }
  51 |     },
  52 |     {
  53 |       "cell_type": "markdown",
  54 |       "source": [
  55 |         "In this notebook you will see how to use an embedding database to store the information that you want to pass to a large language model so that it takes it into account in its responses.\n",
  56 |         "\n",
  57 |         "The information could be your own documents, or whatever was contained in a business knowledge database.\n",
  58 |         "\n",
  59 |         "I have prepared the notebook so that it can work with three different Kaggle datasets, so that it is easy to carry out different tests with different Datasets.\n",
  60 |         "\n",
  61 |         "![RAG Structure](https://github.com/peremartra/Large-Language-Model-Notebooks-Course/blob/main/img/Martra_Figure_2-7.jpg?raw=true)\n"
  62 |       ],
  63 |       "metadata": {
  64 |         "id": "rnSqcL5iOMV-"
  65 |       }
  66 |     },
  67 |     {
  68 |       "cell_type": "markdown",
  69 |       "source": [
  70 |         "#Import Libraries.\n",
  71 |         "To start is necessaryto install some Python packages.\n",
  72 |         "\n",
  73 |         "* **sentence transformers**. This library is necessary to transform the sentences into fixed-length vectors, also know as embeddings.\n",
  74 |         "\n",
  75 |         "* **chromadb**. This is our vector Database. ChromaDB is easy to use and open source, maybe the most used Vector Database used to store embeddings."
  76 |       ],
  77 |       "metadata": {
  78 |         "id": "KDIwyhl5TTGZ"
  79 |       }
  80 |     },
  81 |     {
  82 |       "cell_type": "code",
  83 |       "source": [
  84 |         "!pip install -q transformers==4.41.2"
  85 |       ],
  86 |       "metadata": {
  87 |         "id": "hCD18oMGXI2p"
  88 |       },
  89 |       "execution_count": null,
  90 |       "outputs": []
  91 |     },
  92 |     {
  93 |       "cell_type": "code",
  94 |       "source": [
  95 |         "!pip install -q sentence-transformers==2.2.2\n",
  96 |         "#!pip install -q xformers==0.0.23\n",
  97 |         "!pip install -q chromadb==0.4.20"
  98 |       ],
  99 |       "metadata": {
 100 |         "id": "jMtUKHm6YL5T"
 101 |       },
 102 |       "execution_count": null,
 103 |       "outputs": []
 104 |     },
 105 |     {
 106 |       "cell_type": "markdown",
 107 |       "source": [
 108 |         "I'm sure that you know the next two packages: Numpy and Pandas, maybe the most used python libraries.\n",
 109 |         "\n",
 110 |         "Numpy is a powerful library for numerical computing.\n",
 111 |         "\n",
 112 |         "Pandas is a library for data manipulation"
 113 |       ],
 114 |       "metadata": {
 115 |         "id": "rN_QB7NmYZFj"
 116 |       }
 117 |     },
 118 |     {
 119 |       "cell_type": "code",
 120 |       "source": [
 121 |         "import numpy as np\n",
 122 |         "import pandas as pd"
 123 |       ],
 124 |       "metadata": {
 125 |         "id": "VuDXjIjAYgXm"
 126 |       },
 127 |       "execution_count": null,
 128 |       "outputs": []
 129 |     },
 130 |     {
 131 |       "cell_type": "markdown",
 132 |       "source": [
 133 |         "# Load the Dataset\n",
 134 |         "As you will see the notebook is ready to work with three different Datasets. Just uncomment the lines of the Dataset you want to use.\n",
 135 |         "\n",
 136 |         "I selected Datasets with News. Two of them have just a brief decription of the new, but the other contains the full text.\n",
 137 |         "\n",
 138 |         "As you are working in a memory limited environment, and you can use just a few gb of memory I limited the number of news to use with the variable MAX_NEWS.\n",
 139 |         "\n",
 140 |         "The name of the field containing the text of the new is stored in the variable *DOCUMENT* and the metadata in *TOPIC*"
 141 |       ],
 142 |       "metadata": {
 143 |         "id": "vAPpMqfWYivc"
 144 |       }
 145 |     },
 146 |     {
 147 |       "cell_type": "markdown",
 148 |       "source": [
 149 |         "# Copy Kaggle Dataset\n",
 150 |         "I used the kotartemiy/topic-labeled-news-dataset\n",
 151 |         "https://www.kaggle.com/datasets/kotartemiy/topic-labeled-news-dataset\n",
 152 |         "\n",
 153 |         "Artem Burgara. (2020). R vs. Python: Topic Labeled News Dataset, . Retrieved December 2023, from https://www.kaggle.com/discussions/general/46091.\n",
 154 |         "\n",
 155 |         "But you can ose other datasets, I encourage you to try at least one of these:\n",
 156 |         "* https://www.kaggle.com/datasets/gpreda/bbc-news\n",
 157 |         "* https://www.kaggle.com/datasets/deepanshudalal09/mit-ai-news-published-till-2023\n"
 158 |       ],
 159 |       "metadata": {
 160 |         "id": "QYWQlIo0FzNO"
 161 |       }
 162 |     },
 163 |     {
 164 |       "cell_type": "code",
 165 |       "source": [
 166 |         "from google.colab import drive\n",
 167 |         "drive.mount('/content/drive')"
 168 |       ],
 169 |       "metadata": {
 170 |         "colab": {
 171 |           "base_uri": "https://localhost:8080/"
 172 |         },
 173 |         "id": "xJaT5EwlitXg",
 174 |         "outputId": "2831c0de-927c-417a-f343-8da83cb72015"
 175 |       },
 176 |       "execution_count": null,
 177 |       "outputs": [
 178 |         {
 179 |           "output_type": "stream",
 180 |           "name": "stdout",
 181 |           "text": [
 182 |             "Mounted at /content/drive\n"
 183 |           ]
 184 |         }
 185 |       ]
 186 |     },
 187 |     {
 188 |       "cell_type": "code",
 189 |       "source": [
 190 |         "!pip install kaggle"
 191 |       ],
 192 |       "metadata": {
 193 |         "colab": {
 194 |           "base_uri": "https://localhost:8080/"
 195 |         },
 196 |         "id": "0U9BXNKBi9lR",
 197 |         "outputId": "8e18ddc5-9de5-41ee-8ccd-2e25642f05be"
 198 |       },
 199 |       "execution_count": null,
 200 |       "outputs": [
 201 |         {
 202 |           "output_type": "stream",
 203 |           "name": "stdout",
 204 |           "text": [
 205 |             "Requirement already satisfied: kaggle in /usr/local/lib/python3.10/dist-packages (1.6.14)\n",
 206 |             "Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.10/dist-packages (from kaggle) (1.16.0)\n",
 207 |             "Requirement already satisfied: certifi>=2023.7.22 in /usr/local/lib/python3.10/dist-packages (from kaggle) (2024.6.2)\n",
 208 |             "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.10/dist-packages (from kaggle) (2.8.2)\n",
 209 |             "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from kaggle) (2.31.0)\n",
 210 |             "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from kaggle) (4.66.4)\n",
 211 |             "Requirement already satisfied: python-slugify in /usr/local/lib/python3.10/dist-packages (from kaggle) (8.0.4)\n",
 212 |             "Requirement already satisfied: urllib3 in /usr/local/lib/python3.10/dist-packages (from kaggle) (2.0.7)\n",
 213 |             "Requirement already satisfied: bleach in /usr/local/lib/python3.10/dist-packages (from kaggle) (6.1.0)\n",
 214 |             "Requirement already satisfied: webencodings in /usr/local/lib/python3.10/dist-packages (from bleach->kaggle) (0.5.1)\n",
 215 |             "Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.10/dist-packages (from python-slugify->kaggle) (1.3)\n",
 216 |             "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->kaggle) (3.3.2)\n",
 217 |             "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->kaggle) (3.7)\n"
 218 |           ]
 219 |         }
 220 |       ]
 221 |     },
 222 |     {
 223 |       "cell_type": "code",
 224 |       "source": [
 225 |         "import os\n",
 226 |         "#This directory should contain you kaggle.json file with you key\n",
 227 |         "os.environ['KAGGLE_CONFIG_DIR'] = '/content/drive/MyDrive/kaggle'"
 228 |       ],
 229 |       "metadata": {
 230 |         "id": "GCzM-FdBjHtH"
 231 |       },
 232 |       "execution_count": null,
 233 |       "outputs": []
 234 |     },
 235 |     {
 236 |       "cell_type": "code",
 237 |       "source": [
 238 |         "!kaggle datasets download -d kotartemiy/topic-labeled-news-dataset"
 239 |       ],
 240 |       "metadata": {
 241 |         "colab": {
 242 |           "base_uri": "https://localhost:8080/"
 243 |         },
 244 |         "id": "6ynM9bwdjDyh",
 245 |         "outputId": "a5ee39d4-2276-4175-99df-bf588f04b72a"
 246 |       },
 247 |       "execution_count": null,
 248 |       "outputs": [
 249 |         {
 250 |           "output_type": "stream",
 251 |           "name": "stdout",
 252 |           "text": [
 253 |             "Dataset URL: https://www.kaggle.com/datasets/kotartemiy/topic-labeled-news-dataset\n",
 254 |             "License(s): CC0-1.0\n",
 255 |             "Downloading topic-labeled-news-dataset.zip to /content\n",
 256 |             " 53% 5.00M/9.45M [00:00<00:00, 38.7MB/s]\n",
 257 |             "100% 9.45M/9.45M [00:00<00:00, 51.0MB/s]\n"
 258 |           ]
 259 |         }
 260 |       ]
 261 |     },
 262 |     {
 263 |       "cell_type": "code",
 264 |       "source": [
 265 |         "import zipfile\n",
 266 |         "\n",
 267 |         "# Define the path to your zip file\n",
 268 |         "file_path = '/content/topic-labeled-news-dataset.zip'"
 269 |       ],
 270 |       "metadata": {
 271 |         "id": "GCSzcqOIjlae"
 272 |       },
 273 |       "execution_count": null,
 274 |       "outputs": []
 275 |     },
 276 |     {
 277 |       "cell_type": "code",
 278 |       "source": [
 279 |         "with zipfile.ZipFile(file_path, 'r') as zip_ref:\n",
 280 |         "    zip_ref.extractall('/content/drive/MyDrive/kaggle')"
 281 |       ],
 282 |       "metadata": {
 283 |         "id": "aUdl7-iqkttN"
 284 |       },
 285 |       "execution_count": null,
 286 |       "outputs": []
 287 |     },
 288 |     {
 289 |       "cell_type": "markdown",
 290 |       "source": [
 291 |         "#Loading the Dataset\n",
 292 |         "\n",
 293 |         "Although I've utilized a single dataset for the notebook, I've set it up to facilitate testing with different datasets, available on Kaggle.\n",
 294 |         "\n",
 295 |         "I selected Datasets with News. Two of them have just a brief decription of the new, but the other contains the full text.\n",
 296 |         "\n",
 297 |         "As we are working in a free and limited space, and we can use just 30 gb of memory I limited the number of news to use with the variable MAX_NEWS.\n",
 298 |         "\n",
 299 |         "The name of the field containing the text of the new is stored in the variable DOCUMENT and the metadata in TOPIC.\n"
 300 |       ],
 301 |       "metadata": {
 302 |         "id": "mie-ryBHL9dJ"
 303 |       }
 304 |     },
 305 |     {
 306 |       "cell_type": "code",
 307 |       "source": [
 308 |         "news = pd.read_csv('/content/drive/MyDrive/kaggle/labelled_newscatcher_dataset.csv', sep=';')\n",
 309 |         "MAX_NEWS = 1000\n",
 310 |         "DOCUMENT=\"title\"\n",
 311 |         "TOPIC=\"topic\"\n",
 312 |         "\n",
 313 |         "#Just in case you want to try with a different Dataset.\n",
 314 |         "#news = pd.read_csv('/content/drive/MyDrive/kaggle/bbc_news.csv')\n",
 315 |         "#MAX_NEWS = 1000\n",
 316 |         "#DOCUMENT=\"description\"\n",
 317 |         "#TOPIC=\"title\"\n",
 318 |         "\n",
 319 |         "#news = pd.read_csv('/content/drive/MyDrive/kaggle/mit-ai-news-published-till-2023/articles.csv')\n",
 320 |         "#MAX_NEWS = 100\n",
 321 |         "#DOCUMENT=\"Article Body\"\n",
 322 |         "#TOPIC=\"Article Header\""
 323 |       ],
 324 |       "metadata": {
 325 |         "id": "MeeuZ7tbPrjR"
 326 |       },
 327 |       "execution_count": null,
 328 |       "outputs": []
 329 |     },
 330 |     {
 331 |       "cell_type": "markdown",
 332 |       "source": [
 333 |         "ChromaDB requires that the data has a unique identifier. You can achieve it with the statement below, which will create a new column called **Id**."
 334 |       ],
 335 |       "metadata": {
 336 |         "id": "zGy3ZO4MMY7t"
 337 |       }
 338 |     },
 339 |     {
 340 |       "cell_type": "code",
 341 |       "source": [
 342 |         "news[\"id\"] = news.index\n",
 343 |         "news.head(3)"
 344 |       ],
 345 |       "metadata": {
 346 |         "id": "BPhCplVecPK6",
 347 |         "colab": {
 348 |           "base_uri": "https://localhost:8080/",
 349 |           "height": 424
 350 |         },
 351 |         "outputId": "31901f90-6717-429e-c94a-f34187f9ed76"
 352 |       },
 353 |       "execution_count": null,
 354 |       "outputs": [
 355 |         {
 356 |           "output_type": "execute_result",
 357 |           "data": {
 358 |             "text/plain": [
 359 |               "     topic                                               link          domain  \\\n",
 360 |               "0  SCIENCE  https://www.eurekalert.org/pub_releases/2020-0...  eurekalert.org   \n",
 361 |               "1  SCIENCE  https://www.pulse.ng/news/world/an-irresistibl...        pulse.ng   \n",
 362 |               "2  SCIENCE  https://www.express.co.uk/news/science/1322607...   express.co.uk   \n",
 363 |               "\n",
 364 |               "        published_date                                              title  \\\n",
 365 |               "0  2020-08-06 13:59:45  A closer look at water-splitting's solar fuel ...   \n",
 366 |               "1  2020-08-12 15:14:19  An irresistible scent makes locusts swarm, stu...   \n",
 367 |               "2  2020-08-13 21:01:00  Artificial intelligence warning: AI will know ...   \n",
 368 |               "\n",
 369 |               "  lang  id  \n",
 370 |               "0   en   0  \n",
 371 |               "1   en   1  \n",
 372 |               "2   en   2  "
 373 |             ],
 374 |             "text/html": [
 375 |               "\n",
 376 |               "  <div id=\"df-8f03bc0d-5f0a-4c9e-ae55-fb87d8e88486\" class=\"colab-df-container\">\n",
 377 |               "    <div>\n",
 378 |               "<style scoped>\n",
 379 |               "    .dataframe tbody tr th:only-of-type {\n",
 380 |               "        vertical-align: middle;\n",
 381 |               "    }\n",
 382 |               "\n",
 383 |               "    .dataframe tbody tr th {\n",
 384 |               "        vertical-align: top;\n",
 385 |               "    }\n",
 386 |               "\n",
 387 |               "    .dataframe thead th {\n",
 388 |               "        text-align: right;\n",
 389 |               "    }\n",
 390 |               "</style>\n",
 391 |               "<table border=\"1\" class=\"dataframe\">\n",
 392 |               "  <thead>\n",
 393 |               "    <tr style=\"text-align: right;\">\n",
 394 |               "      <th></th>\n",
 395 |               "      <th>topic</th>\n",
 396 |               "      <th>link</th>\n",
 397 |               "      <th>domain</th>\n",
 398 |               "      <th>published_date</th>\n",
 399 |               "      <th>title</th>\n",
 400 |               "      <th>lang</th>\n",
 401 |               "      <th>id</th>\n",
 402 |               "    </tr>\n",
 403 |               "  </thead>\n",
 404 |               "  <tbody>\n",
 405 |               "    <tr>\n",
 406 |               "      <th>0</th>\n",
 407 |               "      <td>SCIENCE</td>\n",
 408 |               "      <td>https://www.eurekalert.org/pub_releases/2020-0...</td>\n",
 409 |               "      <td>eurekalert.org</td>\n",
 410 |               "      <td>2020-08-06 13:59:45</td>\n",
 411 |               "      <td>A closer look at water-splitting's solar fuel ...</td>\n",
 412 |               "      <td>en</td>\n",
 413 |               "      <td>0</td>\n",
 414 |               "    </tr>\n",
 415 |               "    <tr>\n",
 416 |               "      <th>1</th>\n",
 417 |               "      <td>SCIENCE</td>\n",
 418 |               "      <td>https://www.pulse.ng/news/world/an-irresistibl...</td>\n",
 419 |               "      <td>pulse.ng</td>\n",
 420 |               "      <td>2020-08-12 15:14:19</td>\n",
 421 |               "      <td>An irresistible scent makes locusts swarm, stu...</td>\n",
 422 |               "      <td>en</td>\n",
 423 |               "      <td>1</td>\n",
 424 |               "    </tr>\n",
 425 |               "    <tr>\n",
 426 |               "      <th>2</th>\n",
 427 |               "      <td>SCIENCE</td>\n",
 428 |               "      <td>https://www.express.co.uk/news/science/1322607...</td>\n",
 429 |               "      <td>express.co.uk</td>\n",
 430 |               "      <td>2020-08-13 21:01:00</td>\n",
 431 |               "      <td>Artificial intelligence warning: AI will know ...</td>\n",
 432 |               "      <td>en</td>\n",
 433 |               "      <td>2</td>\n",
 434 |               "    </tr>\n",
 435 |               "  </tbody>\n",
 436 |               "</table>\n",
 437 |               "</div>\n",
 438 |               "    <div class=\"colab-df-buttons\">\n",
 439 |               "\n",
 440 |               "  <div class=\"colab-df-container\">\n",
 441 |               "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8f03bc0d-5f0a-4c9e-ae55-fb87d8e88486')\"\n",
 442 |               "            title=\"Convert this dataframe to an interactive table.\"\n",
 443 |               "            style=\"display:none;\">\n",
 444 |               "\n",
 445 |               "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
 446 |               "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
 447 |               "  </svg>\n",
 448 |               "    </button>\n",
 449 |               "\n",
 450 |               "  <style>\n",
 451 |               "    .colab-df-container {\n",
 452 |               "      display:flex;\n",
 453 |               "      gap: 12px;\n",
 454 |               "    }\n",
 455 |               "\n",
 456 |               "    .colab-df-convert {\n",
 457 |               "      background-color: #E8F0FE;\n",
 458 |               "      border: none;\n",
 459 |               "      border-radius: 50%;\n",
 460 |               "      cursor: pointer;\n",
 461 |               "      display: none;\n",
 462 |               "      fill: #1967D2;\n",
 463 |               "      height: 32px;\n",
 464 |               "      padding: 0 0 0 0;\n",
 465 |               "      width: 32px;\n",
 466 |               "    }\n",
 467 |               "\n",
 468 |               "    .colab-df-convert:hover {\n",
 469 |               "      background-color: #E2EBFA;\n",
 470 |               "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
 471 |               "      fill: #174EA6;\n",
 472 |               "    }\n",
 473 |               "\n",
 474 |               "    .colab-df-buttons div {\n",
 475 |               "      margin-bottom: 4px;\n",
 476 |               "    }\n",
 477 |               "\n",
 478 |               "    [theme=dark] .colab-df-convert {\n",
 479 |               "      background-color: #3B4455;\n",
 480 |               "      fill: #D2E3FC;\n",
 481 |               "    }\n",
 482 |               "\n",
 483 |               "    [theme=dark] .colab-df-convert:hover {\n",
 484 |               "      background-color: #434B5C;\n",
 485 |               "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
 486 |               "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
 487 |               "      fill: #FFFFFF;\n",
 488 |               "    }\n",
 489 |               "  </style>\n",
 490 |               "\n",
 491 |               "    <script>\n",
 492 |               "      const buttonEl =\n",
 493 |               "        document.querySelector('#df-8f03bc0d-5f0a-4c9e-ae55-fb87d8e88486 button.colab-df-convert');\n",
 494 |               "      buttonEl.style.display =\n",
 495 |               "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
 496 |               "\n",
 497 |               "      async function convertToInteractive(key) {\n",
 498 |               "        const element = document.querySelector('#df-8f03bc0d-5f0a-4c9e-ae55-fb87d8e88486');\n",
 499 |               "        const dataTable =\n",
 500 |               "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
 501 |               "                                                    [key], {});\n",
 502 |               "        if (!dataTable) return;\n",
 503 |               "\n",
 504 |               "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
 505 |               "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
 506 |               "          + ' to learn more about interactive tables.';\n",
 507 |               "        element.innerHTML = '';\n",
 508 |               "        dataTable['output_type'] = 'display_data';\n",
 509 |               "        await google.colab.output.renderOutput(dataTable, element);\n",
 510 |               "        const docLink = document.createElement('div');\n",
 511 |               "        docLink.innerHTML = docLinkHtml;\n",
 512 |               "        element.appendChild(docLink);\n",
 513 |               "      }\n",
 514 |               "    </script>\n",
 515 |               "  </div>\n",
 516 |               "\n",
 517 |               "\n",
 518 |               "<div id=\"df-1e4f69e6-e7b0-48b8-8e7b-0bebf996c922\">\n",
 519 |               "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-1e4f69e6-e7b0-48b8-8e7b-0bebf996c922')\"\n",
 520 |               "            title=\"Suggest charts\"\n",
 521 |               "            style=\"display:none;\">\n",
 522 |               "\n",
 523 |               "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
 524 |               "     width=\"24px\">\n",
 525 |               "    <g>\n",
 526 |               "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
 527 |               "    </g>\n",
 528 |               "</svg>\n",
 529 |               "  </button>\n",
 530 |               "\n",
 531 |               "<style>\n",
 532 |               "  .colab-df-quickchart {\n",
 533 |               "      --bg-color: #E8F0FE;\n",
 534 |               "      --fill-color: #1967D2;\n",
 535 |               "      --hover-bg-color: #E2EBFA;\n",
 536 |               "      --hover-fill-color: #174EA6;\n",
 537 |               "      --disabled-fill-color: #AAA;\n",
 538 |               "      --disabled-bg-color: #DDD;\n",
 539 |               "  }\n",
 540 |               "\n",
 541 |               "  [theme=dark] .colab-df-quickchart {\n",
 542 |               "      --bg-color: #3B4455;\n",
 543 |               "      --fill-color: #D2E3FC;\n",
 544 |               "      --hover-bg-color: #434B5C;\n",
 545 |               "      --hover-fill-color: #FFFFFF;\n",
 546 |               "      --disabled-bg-color: #3B4455;\n",
 547 |               "      --disabled-fill-color: #666;\n",
 548 |               "  }\n",
 549 |               "\n",
 550 |               "  .colab-df-quickchart {\n",
 551 |               "    background-color: var(--bg-color);\n",
 552 |               "    border: none;\n",
 553 |               "    border-radius: 50%;\n",
 554 |               "    cursor: pointer;\n",
 555 |               "    display: none;\n",
 556 |               "    fill: var(--fill-color);\n",
 557 |               "    height: 32px;\n",
 558 |               "    padding: 0;\n",
 559 |               "    width: 32px;\n",
 560 |               "  }\n",
 561 |               "\n",
 562 |               "  .colab-df-quickchart:hover {\n",
 563 |               "    background-color: var(--hover-bg-color);\n",
 564 |               "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
 565 |               "    fill: var(--button-hover-fill-color);\n",
 566 |               "  }\n",
 567 |               "\n",
 568 |               "  .colab-df-quickchart-complete:disabled,\n",
 569 |               "  .colab-df-quickchart-complete:disabled:hover {\n",
 570 |               "    background-color: var(--disabled-bg-color);\n",
 571 |               "    fill: var(--disabled-fill-color);\n",
 572 |               "    box-shadow: none;\n",
 573 |               "  }\n",
 574 |               "\n",
 575 |               "  .colab-df-spinner {\n",
 576 |               "    border: 2px solid var(--fill-color);\n",
 577 |               "    border-color: transparent;\n",
 578 |               "    border-bottom-color: var(--fill-color);\n",
 579 |               "    animation:\n",
 580 |               "      spin 1s steps(1) infinite;\n",
 581 |               "  }\n",
 582 |               "\n",
 583 |               "  @keyframes spin {\n",
 584 |               "    0% {\n",
 585 |               "      border-color: transparent;\n",
 586 |               "      border-bottom-color: var(--fill-color);\n",
 587 |               "      border-left-color: var(--fill-color);\n",
 588 |               "    }\n",
 589 |               "    20% {\n",
 590 |               "      border-color: transparent;\n",
 591 |               "      border-left-color: var(--fill-color);\n",
 592 |               "      border-top-color: var(--fill-color);\n",
 593 |               "    }\n",
 594 |               "    30% {\n",
 595 |               "      border-color: transparent;\n",
 596 |               "      border-left-color: var(--fill-color);\n",
 597 |               "      border-top-color: var(--fill-color);\n",
 598 |               "      border-right-color: var(--fill-color);\n",
 599 |               "    }\n",
 600 |               "    40% {\n",
 601 |               "      border-color: transparent;\n",
 602 |               "      border-right-color: var(--fill-color);\n",
 603 |               "      border-top-color: var(--fill-color);\n",
 604 |               "    }\n",
 605 |               "    60% {\n",
 606 |               "      border-color: transparent;\n",
 607 |               "      border-right-color: var(--fill-color);\n",
 608 |               "    }\n",
 609 |               "    80% {\n",
 610 |               "      border-color: transparent;\n",
 611 |               "      border-right-color: var(--fill-color);\n",
 612 |               "      border-bottom-color: var(--fill-color);\n",
 613 |               "    }\n",
 614 |               "    90% {\n",
 615 |               "      border-color: transparent;\n",
 616 |               "      border-bottom-color: var(--fill-color);\n",
 617 |               "    }\n",
 618 |               "  }\n",
 619 |               "</style>\n",
 620 |               "\n",
 621 |               "  <script>\n",
 622 |               "    async function quickchart(key) {\n",
 623 |               "      const quickchartButtonEl =\n",
 624 |               "        document.querySelector('#' + key + ' button');\n",
 625 |               "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
 626 |               "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
 627 |               "      try {\n",
 628 |               "        const charts = await google.colab.kernel.invokeFunction(\n",
 629 |               "            'suggestCharts', [key], {});\n",
 630 |               "      } catch (error) {\n",
 631 |               "        console.error('Error during call to suggestCharts:', error);\n",
 632 |               "      }\n",
 633 |               "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
 634 |               "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
 635 |               "    }\n",
 636 |               "    (() => {\n",
 637 |               "      let quickchartButtonEl =\n",
 638 |               "        document.querySelector('#df-1e4f69e6-e7b0-48b8-8e7b-0bebf996c922 button');\n",
 639 |               "      quickchartButtonEl.style.display =\n",
 640 |               "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
 641 |               "    })();\n",
 642 |               "  </script>\n",
 643 |               "</div>\n",
 644 |               "\n",
 645 |               "    </div>\n",
 646 |               "  </div>\n"
 647 |             ],
 648 |             "application/vnd.google.colaboratory.intrinsic+json": {
 649 |               "type": "dataframe",
 650 |               "variable_name": "news"
 651 |             }
 652 |           },
 653 |           "metadata": {},
 654 |           "execution_count": 15
 655 |         }
 656 |       ]
 657 |     },
 658 |     {
 659 |       "cell_type": "code",
 660 |       "source": [
 661 |         "#Because it is just a example we select a small portion of News.\n",
 662 |         "subset_news = news.head(MAX_NEWS)"
 663 |       ],
 664 |       "metadata": {
 665 |         "id": "elYy8a0OTJaf"
 666 |       },
 667 |       "execution_count": null,
 668 |       "outputs": []
 669 |     },
 670 |     {
 671 |       "cell_type": "markdown",
 672 |       "source": [
 673 |         "# Import and configure the Vector Database\n",
 674 |         "You are going to use ChromaDB, the most popular OpenSource embedding Database.\n",
 675 |         "\n",
 676 |         "First you need to import ChromaDB, and after that import the **Settings** class from **chromadb.config** module. This class allows to change the setting for the ChromaDB system, and customize its behavior."
 677 |       ],
 678 |       "metadata": {
 679 |         "id": "S8tDSbZ3MxZP"
 680 |       }
 681 |     },
 682 |     {
 683 |       "cell_type": "code",
 684 |       "source": [
 685 |         "import chromadb\n",
 686 |         "from chromadb.config import Settings"
 687 |       ],
 688 |       "metadata": {
 689 |         "id": "cXYYHBJzMl5n"
 690 |       },
 691 |       "execution_count": null,
 692 |       "outputs": []
 693 |     },
 694 |     {
 695 |       "cell_type": "markdown",
 696 |       "source": [
 697 |         "Now you need to create the seetings object calling the Settings function imported previously. The object is stored in the variable **settings_chroma**.\n",
 698 |         "\n",
 699 |         "You need to inform two parameters\n",
 700 |         "\n",
 701 |         "* **chroma_db_impl**. Here you must specify the database implementation and the format how store the data. I choose **duckdb**, because his high-performace. It operate primarly in memory. And is fully compatible with SQL. The store format **parquet** is good for tabular data. With good compression rates and performance.\n",
 702 |         "\n",
 703 |         "* **persist_directory**: It just contains the directory where the data will be stored. Is possible work without a directory and the data will be stored in memory without persistece, but some cloud providers or platforms like Kaggle dosn't support that."
 704 |       ],
 705 |       "metadata": {
 706 |         "id": "tmFsTy9XOPla"
 707 |       }
 708 |     },
 709 |     {
 710 |       "cell_type": "code",
 711 |       "source": [
 712 |         "#OLD VERSION\n",
 713 |         "#settings_chroma = Settings(chroma_db_impl=\"duckdb+parquet\",\n",
 714 |         "#                          persist_directory='./input')\n",
 715 |         "#chroma_client = chromadb.Client(settings_chroma)\n",
 716 |         "\n",
 717 |         "#NEW VERSION => 0.40\n",
 718 |         "chroma_client = chromadb.PersistentClient(path=\"/content/drive/MyDrive/chromadb\")"
 719 |       ],
 720 |       "metadata": {
 721 |         "id": "tAbROIjCONv7"
 722 |       },
 723 |       "execution_count": null,
 724 |       "outputs": []
 725 |     },
 726 |     {
 727 |       "cell_type": "markdown",
 728 |       "source": [
 729 |         "# Filling and Querying the ChromaDB Database\n",
 730 |         "The Data in ChromaDB is stored in collections. If the collection previously exist is necessary to delete it.\n",
 731 |         "\n",
 732 |         "In the next lines, the collection is created by calling the ***create_collection*** function in the ***chroma_client*** created above."
 733 |       ],
 734 |       "metadata": {
 735 |         "id": "vxlh6QfiSK-p"
 736 |       }
 737 |     },
 738 |     {
 739 |       "cell_type": "code",
 740 |       "source": [
 741 |         "from datetime import datetime"
 742 |       ],
 743 |       "metadata": {
 744 |         "id": "G4O2UuGzW5mC"
 745 |       },
 746 |       "execution_count": null,
 747 |       "outputs": []
 748 |     },
 749 |     {
 750 |       "cell_type": "code",
 751 |       "source": [
 752 |         "collection_name = \"news_collection\"+datetime.now().strftime(\"%s\")\n",
 753 |         "if len(chroma_client.list_collections()) > 0 and collection_name in [chroma_client.list_collections()[0].name]:\n",
 754 |         "        chroma_client.delete_collection(name=collection_name)\n",
 755 |         "\n",
 756 |         "collection = chroma_client.create_collection(name=collection_name)"
 757 |       ],
 758 |       "metadata": {
 759 |         "id": "6HOnr43oO1vG"
 760 |       },
 761 |       "execution_count": null,
 762 |       "outputs": []
 763 |     },
 764 |     {
 765 |       "cell_type": "markdown",
 766 |       "source": [
 767 |         "It's time to add the data to the collection. Using the function ***add*** you should inform, at least ***documents***, ***metadatas*** and ***ids***.\n",
 768 |         "* In the **document** the full news text is stored, remember that it is contained in a different column for each Dataset.\n",
 769 |         "* In **metadatas**, we can inform a list of topics.\n",
 770 |         "* In **id** an unique identificator for each row must be informed. It MUST be unique! I'm creating the ID using the range of MAX_NEWS."
 771 |       ],
 772 |       "metadata": {
 773 |         "id": "C-_0xWxQSfwv"
 774 |       }
 775 |     },
 776 |     {
 777 |       "cell_type": "code",
 778 |       "source": [
 779 |         "collection.add(\n",
 780 |         "    documents=subset_news[DOCUMENT].tolist(),\n",
 781 |         "    metadatas=[{TOPIC: topic} for topic in subset_news[TOPIC].tolist()],\n",
 782 |         "    ids=[f\"id{x}\" for x in range(MAX_NEWS)],\n",
 783 |         ")"
 784 |       ],
 785 |       "metadata": {
 786 |         "id": "xuAz6R0gSZho",
 787 |         "colab": {
 788 |           "base_uri": "https://localhost:8080/"
 789 |         },
 790 |         "outputId": "a47379c8-e4a5-49fd-c055-f7be35196ca0"
 791 |       },
 792 |       "execution_count": null,
 793 |       "outputs": [
 794 |         {
 795 |           "output_type": "stream",
 796 |           "name": "stderr",
 797 |           "text": [
 798 |             "/root/.cache/chroma/onnx_models/all-MiniLM-L6-v2/onnx.tar.gz: 100%|██████████| 79.3M/79.3M [00:03<00:00, 21.7MiB/s]\n"
 799 |           ]
 800 |         }
 801 |       ]
 802 |     },
 803 |     {
 804 |       "cell_type": "code",
 805 |       "source": [
 806 |         "results = collection.query(query_texts=[\"laptop\"], n_results=10 )\n",
 807 |         "\n",
 808 |         "print(results)"
 809 |       ],
 810 |       "metadata": {
 811 |         "colab": {
 812 |           "base_uri": "https://localhost:8080/"
 813 |         },
 814 |         "id": "IpofynmFTCKm",
 815 |         "outputId": "8fb85c27-3993-4703-970b-a130bb69290a"
 816 |       },
 817 |       "execution_count": null,
 818 |       "outputs": [
 819 |         {
 820 |           "output_type": "stream",
 821 |           "name": "stdout",
 822 |           "text": [
 823 |             "{'ids': [['id173', 'id829', 'id117', 'id535', 'id141', 'id218', 'id390', 'id273', 'id56', 'id900']], 'distances': [[0.8593594431877136, 1.0294400453567505, 1.0793331861495972, 1.093001127243042, 1.1329681873321533, 1.2130440473556519, 1.214331865310669, 1.2164140939712524, 1.2220635414123535, 1.2754170894622803]], 'metadatas': [[{'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}]], 'embeddings': None, 'documents': [['The Legendary Toshiba is Officially Done With Making Laptops', '3 gaming laptop deals you can’t afford to miss today', 'Lenovo and HP control half of the global laptop market', 'Asus ROG Zephyrus G14 gaming laptop announced in India', 'Acer Swift 3 featuring a 10th-generation Intel Ice Lake CPU, 2K screen, and more launched in India for INR 64999 (US$865)', \"Apple's Next MacBook Could Be the Cheapest in Company's History\", \"Features of Huawei's Desktop Computer Revealed\", 'Redmi to launch its first gaming laptop on August 14: Here are all the details', 'Toshiba shuts the lid on laptops after 35 years', 'This is the cheapest Windows PC by a mile and it even has a spare SSD slot']], 'uris': None, 'data': None}\n"
 824 |           ]
 825 |         }
 826 |       ]
 827 |     },
 828 |     {
 829 |       "cell_type": "markdown",
 830 |       "source": [
 831 |         "#Vector MAP"
 832 |       ],
 833 |       "metadata": {
 834 |         "id": "b4PDwcbHcQqO"
 835 |       }
 836 |     },
 837 |     {
 838 |       "cell_type": "code",
 839 |       "source": [
 840 |         "import matplotlib.pyplot as plt\n",
 841 |         "from sklearn.decomposition import PCA"
 842 |       ],
 843 |       "metadata": {
 844 |         "id": "X8s8H-49cNmb"
 845 |       },
 846 |       "execution_count": null,
 847 |       "outputs": []
 848 |     },
 849 |     {
 850 |       "cell_type": "code",
 851 |       "source": [
 852 |         "getado = collection.get(ids=\"id141\",\n",
 853 |         "                       include=[\"documents\", \"embeddings\"])"
 854 |       ],
 855 |       "metadata": {
 856 |         "id": "xz9rF11rcP2r"
 857 |       },
 858 |       "execution_count": null,
 859 |       "outputs": []
 860 |     },
 861 |     {
 862 |       "cell_type": "code",
 863 |       "source": [
 864 |         "word_vectors = getado[\"embeddings\"]\n",
 865 |         "word_list = getado[\"documents\"]\n",
 866 |         "word_vectors"
 867 |       ],
 868 |       "metadata": {
 869 |         "colab": {
 870 |           "base_uri": "https://localhost:8080/"
 871 |         },
 872 |         "id": "kLJtGzfecXAO",
 873 |         "outputId": "a7388dee-201c-42d5-99a9-1742cc506301"
 874 |       },
 875 |       "execution_count": null,
 876 |       "outputs": [
 877 |         {
 878 |           "output_type": "execute_result",
 879 |           "data": {
 880 |             "text/plain": [
 881 |               "[[-0.0808560848236084,\n",
 882 |               "  -0.049963705241680145,\n",
 883 |               "  -0.023777484893798828,\n",
 884 |               "  -0.011053602211177349,\n",
 885 |               "  0.02665771171450615,\n",
 886 |               "  -0.04479333013296127,\n",
 887 |               "  -0.02889663353562355,\n",
 888 |               "  0.026656104251742363,\n",
 889 |               "  0.0014397227205336094,\n",
 890 |               "  -0.016407841816544533,\n",
 891 |               "  0.0653492733836174,\n",
 892 |               "  -0.06901992857456207,\n",
 893 |               "  -0.05748078227043152,\n",
 894 |               "  0.010111615061759949,\n",
 895 |               "  0.05043035000562668,\n",
 896 |               "  -0.002057764446362853,\n",
 897 |               "  0.07256408035755157,\n",
 898 |               "  -0.12437368929386139,\n",
 899 |               "  0.010659442283213139,\n",
 900 |               "  -0.10942046344280243,\n",
 901 |               "  -0.01143240462988615,\n",
 902 |               "  -0.010376011952757835,\n",
 903 |               "  -0.020610831677913666,\n",
 904 |               "  -0.024394094944000244,\n",
 905 |               "  0.07828476279973984,\n",
 906 |               "  0.005820558872073889,\n",
 907 |               "  0.023317726328969002,\n",
 908 |               "  -0.08243829756975174,\n",
 909 |               "  -0.02726505883038044,\n",
 910 |               "  0.0046674772165715694,\n",
 911 |               "  0.004340188577771187,\n",
 912 |               "  0.03252805024385452,\n",
 913 |               "  -0.026030974462628365,\n",
 914 |               "  0.07963905483484268,\n",
 915 |               "  0.042182061821222305,\n",
 916 |               "  -0.12119994312524796,\n",
 917 |               "  0.04907083883881569,\n",
 918 |               "  -0.07625846564769745,\n",
 919 |               "  0.04331624507904053,\n",
 920 |               "  -0.08360457420349121,\n",
 921 |               "  -0.07140401750802994,\n",
 922 |               "  -0.018792513757944107,\n",
 923 |               "  0.036049388349056244,\n",
 924 |               "  0.042845625430345535,\n",
 925 |               "  0.025760438293218613,\n",
 926 |               "  0.03972514718770981,\n",
 927 |               "  -0.007091294974088669,\n",
 928 |               "  0.035189948976039886,\n",
 929 |               "  0.027369096875190735,\n",
 930 |               "  0.009289839304983616,\n",
 931 |               "  -0.03916166350245476,\n",
 932 |               "  -0.037401288747787476,\n",
 933 |               "  -0.03369569778442383,\n",
 934 |               "  -0.06543102860450745,\n",
 935 |               "  0.01919756457209587,\n",
 936 |               "  -0.009083813056349754,\n",
 937 |               "  0.022507958114147186,\n",
 938 |               "  -0.04346753656864166,\n",
 939 |               "  0.03663364797830582,\n",
 940 |               "  0.09003791213035583,\n",
 941 |               "  0.037535011768341064,\n",
 942 |               "  -0.04698672145605087,\n",
 943 |               "  -0.021450482308864594,\n",
 944 |               "  0.049021799117326736,\n",
 945 |               "  0.0008767120307311416,\n",
 946 |               "  -0.0491013377904892,\n",
 947 |               "  0.019651763141155243,\n",
 948 |               "  -0.11427141726016998,\n",
 949 |               "  -0.0041212718933820724,\n",
 950 |               "  -0.059363994747400284,\n",
 951 |               "  0.08288168162107468,\n",
 952 |               "  -0.015205126255750656,\n",
 953 |               "  0.05800360441207886,\n",
 954 |               "  -0.009099257178604603,\n",
 955 |               "  -0.06618720293045044,\n",
 956 |               "  -0.04995487257838249,\n",
 957 |               "  0.051422737538814545,\n",
 958 |               "  -0.030697545036673546,\n",
 959 |               "  -0.00599612295627594,\n",
 960 |               "  -0.017441147938370705,\n",
 961 |               "  -0.0067165326327085495,\n",
 962 |               "  -0.026863766834139824,\n",
 963 |               "  0.009797872975468636,\n",
 964 |               "  0.012698033824563026,\n",
 965 |               "  -0.017240023240447044,\n",
 966 |               "  -0.040722113102674484,\n",
 967 |               "  0.02619202248752117,\n",
 968 |               "  -0.036331336945295334,\n",
 969 |               "  -0.005993332713842392,\n",
 970 |               "  -0.03835161030292511,\n",
 971 |               "  -0.0025671017356216908,\n",
 972 |               "  0.017816301435232162,\n",
 973 |               "  0.015638208016753197,\n",
 974 |               "  -0.002425231970846653,\n",
 975 |               "  -0.04693610966205597,\n",
 976 |               "  0.014938564039766788,\n",
 977 |               "  0.057856056839227676,\n",
 978 |               "  -0.04266524314880371,\n",
 979 |               "  -0.051958806812763214,\n",
 980 |               "  0.07029527425765991,\n",
 981 |               "  0.0174096692353487,\n",
 982 |               "  0.01044029463082552,\n",
 983 |               "  0.06522297114133835,\n",
 984 |               "  -0.0005443890695460141,\n",
 985 |               "  -0.0072739566676318645,\n",
 986 |               "  -0.01764906942844391,\n",
 987 |               "  -0.01343501266092062,\n",
 988 |               "  0.04487789049744606,\n",
 989 |               "  -0.04048682376742363,\n",
 990 |               "  0.05065847188234329,\n",
 991 |               "  -0.006964817643165588,\n",
 992 |               "  -0.039026305079460144,\n",
 993 |               "  -0.06116586551070213,\n",
 994 |               "  0.009619001299142838,\n",
 995 |               "  -0.01937313936650753,\n",
 996 |               "  -0.07259991019964218,\n",
 997 |               "  -0.05417889729142189,\n",
 998 |               "  -0.012206215411424637,\n",
 999 |               "  0.14574117958545685,\n",
1000 |               "  0.07729611545801163,\n",
1001 |               "  -0.03489774465560913,\n",
1002 |               "  0.02000734768807888,\n",
1003 |               "  -0.00971849262714386,\n",
1004 |               "  -0.0023303572088479996,\n",
1005 |               "  -0.08877687901258469,\n",
1006 |               "  0.007430689875036478,\n",
1007 |               "  -0.022453872486948967,\n",
1008 |               "  2.939167320216669e-33,\n",
1009 |               "  -0.03226035088300705,\n",
1010 |               "  0.02646695077419281,\n",
1011 |               "  -0.06441568583250046,\n",
1012 |               "  -0.10495101660490036,\n",
1013 |               "  0.007915517315268517,\n",
1014 |               "  -0.056241028010845184,\n",
1015 |               "  0.0600648857653141,\n",
1016 |               "  0.020356711000204086,\n",
1017 |               "  0.008408062160015106,\n",
1018 |               "  0.00394280394539237,\n",
1019 |               "  -0.07579407840967178,\n",
1020 |               "  -0.059283699840307236,\n",
1021 |               "  -0.07090269774198532,\n",
1022 |               "  0.047602903097867966,\n",
1023 |               "  0.12866826355457306,\n",
1024 |               "  -0.09145348519086838,\n",
1025 |               "  -0.06453359127044678,\n",
1026 |               "  -0.022725841030478477,\n",
1027 |               "  -0.007938683032989502,\n",
1028 |               "  0.08817118406295776,\n",
1029 |               "  0.030397845432162285,\n",
1030 |               "  -0.08102215081453323,\n",
1031 |               "  -0.004711293615400791,\n",
1032 |               "  0.008305762894451618,\n",
1033 |               "  -0.008205698803067207,\n",
1034 |               "  0.044841013848781586,\n",
1035 |               "  0.003640854964032769,\n",
1036 |               "  -0.015380569733679295,\n",
1037 |               "  0.06941366195678711,\n",
1038 |               "  0.026241688057780266,\n",
1039 |               "  0.042122967541217804,\n",
1040 |               "  -0.022600207477808,\n",
1041 |               "  -0.004685663152486086,\n",
1042 |               "  -0.08609002083539963,\n",
1043 |               "  -0.0017022470710799098,\n",
1044 |               "  -0.036475248634815216,\n",
1045 |               "  0.03104810230433941,\n",
1046 |               "  -0.07396797835826874,\n",
1047 |               "  0.0006451740628108382,\n",
1048 |               "  0.057689860463142395,\n",
1049 |               "  -0.03433110564947128,\n",
1050 |               "  0.10101553797721863,\n",
1051 |               "  -0.07978133112192154,\n",
1052 |               "  -0.01770963706076145,\n",
1053 |               "  0.03582654148340225,\n",
1054 |               "  0.07763752341270447,\n",
1055 |               "  0.007194378413259983,\n",
1056 |               "  0.05085881054401398,\n",
1057 |               "  0.03166932985186577,\n",
1058 |               "  -0.050254058092832565,\n",
1059 |               "  -0.1192030981183052,\n",
1060 |               "  -0.002129735192283988,\n",
1061 |               "  -0.0107394028455019,\n",
1062 |               "  -0.048253513872623444,\n",
1063 |               "  0.05193442106246948,\n",
1064 |               "  -0.0033458012621849775,\n",
1065 |               "  0.045708633959293365,\n",
1066 |               "  -0.0063198041170835495,\n",
1067 |               "  0.13073934614658356,\n",
1068 |               "  0.05694258213043213,\n",
1069 |               "  -0.1031755656003952,\n",
1070 |               "  -0.021950311958789825,\n",
1071 |               "  -0.05125086382031441,\n",
1072 |               "  -0.006670759059488773,\n",
1073 |               "  -0.04244102165102959,\n",
1074 |               "  0.07283089309930801,\n",
1075 |               "  0.08295755833387375,\n",
1076 |               "  -0.01451050490140915,\n",
1077 |               "  -0.050436634570360184,\n",
1078 |               "  -0.0063996752724051476,\n",
1079 |               "  -0.05111794173717499,\n",
1080 |               "  -0.06090034544467926,\n",
1081 |               "  0.12017730623483658,\n",
1082 |               "  -0.007007972337305546,\n",
1083 |               "  -0.018361017107963562,\n",
1084 |               "  0.05737388879060745,\n",
1085 |               "  -0.06952648609876633,\n",
1086 |               "  -0.0329073891043663,\n",
1087 |               "  -0.046019524335861206,\n",
1088 |               "  -0.04026426374912262,\n",
1089 |               "  -0.03974202275276184,\n",
1090 |               "  0.04068472236394882,\n",
1091 |               "  0.06414211541414261,\n",
1092 |               "  0.08900655806064606,\n",
1093 |               "  -0.00060390739236027,\n",
1094 |               "  0.06864351779222488,\n",
1095 |               "  -0.04542217403650284,\n",
1096 |               "  -0.012834936380386353,\n",
1097 |               "  0.014704265631735325,\n",
1098 |               "  0.08229125291109085,\n",
1099 |               "  -0.011864698491990566,\n",
1100 |               "  -0.007569513283669949,\n",
1101 |               "  0.029478946700692177,\n",
1102 |               "  -0.017556926235556602,\n",
1103 |               "  0.03026776760816574,\n",
1104 |               "  -3.4444798318695305e-33,\n",
1105 |               "  0.013093586079776287,\n",
1106 |               "  -0.05642778053879738,\n",
1107 |               "  -0.05393407866358757,\n",
1108 |               "  0.02010548673570156,\n",
1109 |               "  0.002186316065490246,\n",
1110 |               "  0.021474365144968033,\n",
1111 |               "  -0.01675189472734928,\n",
1112 |               "  0.11519582569599152,\n",
1113 |               "  0.009137553162872791,\n",
1114 |               "  0.002790238708257675,\n",
1115 |               "  -0.028134936466813087,\n",
1116 |               "  0.08885099738836288,\n",
1117 |               "  0.07289770990610123,\n",
1118 |               "  0.02972547896206379,\n",
1119 |               "  0.03313830494880676,\n",
1120 |               "  -0.03830660134553909,\n",
1121 |               "  -0.015428598038852215,\n",
1122 |               "  -0.025887154042720795,\n",
1123 |               "  0.02991276979446411,\n",
1124 |               "  0.021465003490447998,\n",
1125 |               "  0.055984679609537125,\n",
1126 |               "  0.02411733567714691,\n",
1127 |               "  0.013602275401353836,\n",
1128 |               "  0.001853855443187058,\n",
1129 |               "  0.046712420880794525,\n",
1130 |               "  0.019653165712952614,\n",
1131 |               "  -0.0611024871468544,\n",
1132 |               "  0.006502089090645313,\n",
1133 |               "  0.0441303625702858,\n",
1134 |               "  -0.030408691614866257,\n",
1135 |               "  0.003392198821529746,\n",
1136 |               "  -0.07827484607696533,\n",
1137 |               "  0.08200996369123459,\n",
1138 |               "  0.021945785731077194,\n",
1139 |               "  -0.04214276745915413,\n",
1140 |               "  -0.011046931147575378,\n",
1141 |               "  0.11221751570701599,\n",
1142 |               "  -0.03547649085521698,\n",
1143 |               "  -0.02034923806786537,\n",
1144 |               "  0.05863974243402481,\n",
1145 |               "  0.061948224902153015,\n",
1146 |               "  0.004163042642176151,\n",
1147 |               "  0.025604233145713806,\n",
1148 |               "  0.07054829597473145,\n",
1149 |               "  0.026611244305968285,\n",
1150 |               "  0.03901861235499382,\n",
1151 |               "  -0.0028351128567010164,\n",
1152 |               "  -0.013862921856343746,\n",
1153 |               "  -0.03907373547554016,\n",
1154 |               "  -0.10136910527944565,\n",
1155 |               "  0.015273002907633781,\n",
1156 |               "  -0.0636049285531044,\n",
1157 |               "  0.009429153054952621,\n",
1158 |               "  -0.03778139874339104,\n",
1159 |               "  -0.07231791317462921,\n",
1160 |               "  -0.05598380044102669,\n",
1161 |               "  0.0146296676248312,\n",
1162 |               "  0.006302482448518276,\n",
1163 |               "  0.036601871252059937,\n",
1164 |               "  -0.07091113924980164,\n",
1165 |               "  0.03455745428800583,\n",
1166 |               "  0.021303288638591766,\n",
1167 |               "  0.018934456631541252,\n",
1168 |               "  0.01931314915418625,\n",
1169 |               "  0.007075129076838493,\n",
1170 |               "  0.018622223287820816,\n",
1171 |               "  0.05089221149682999,\n",
1172 |               "  0.03337225690484047,\n",
1173 |               "  -0.008652801625430584,\n",
1174 |               "  -0.023814190179109573,\n",
1175 |               "  -0.05765106528997421,\n",
1176 |               "  -0.10482978075742722,\n",
1177 |               "  0.013372018001973629,\n",
1178 |               "  -0.01933068409562111,\n",
1179 |               "  -0.016363972797989845,\n",
1180 |               "  0.04313588887453079,\n",
1181 |               "  -0.019315389916300774,\n",
1182 |               "  0.042876679450273514,\n",
1183 |               "  0.07228625565767288,\n",
1184 |               "  -0.004229306243360043,\n",
1185 |               "  0.025138597935438156,\n",
1186 |               "  0.07576548308134079,\n",
1187 |               "  0.033386338502168655,\n",
1188 |               "  0.022974086925387383,\n",
1189 |               "  0.07685324549674988,\n",
1190 |               "  -0.05225932598114014,\n",
1191 |               "  0.043289393186569214,\n",
1192 |               "  -0.012385099194943905,\n",
1193 |               "  -0.036590274423360825,\n",
1194 |               "  -0.012774484232068062,\n",
1195 |               "  -0.046136267483234406,\n",
1196 |               "  0.051983099430799484,\n",
1197 |               "  -0.06112852692604065,\n",
1198 |               "  -0.0003355621884111315,\n",
1199 |               "  -0.006177410017699003,\n",
1200 |               "  -2.835624535180159e-08,\n",
1201 |               "  0.06613793969154358,\n",
1202 |               "  0.011798004619777203,\n",
1203 |               "  0.037113726139068604,\n",
1204 |               "  0.04696718230843544,\n",
1205 |               "  0.043035347014665604,\n",
1206 |               "  -0.09251100569963455,\n",
1207 |               "  0.046135202050209045,\n",
1208 |               "  0.08055239915847778,\n",
1209 |               "  0.10685128718614578,\n",
1210 |               "  -0.007414942141622305,\n",
1211 |               "  -0.0413203239440918,\n",
1212 |               "  -0.08303714543581009,\n",
1213 |               "  -0.018203390762209892,\n",
1214 |               "  0.013738766312599182,\n",
1215 |               "  0.043958164751529694,\n",
1216 |               "  0.037307366728782654,\n",
1217 |               "  0.03315778449177742,\n",
1218 |               "  0.0881715714931488,\n",
1219 |               "  0.0019880577456206083,\n",
1220 |               "  -0.07271483540534973,\n",
1221 |               "  0.0230609979480505,\n",
1222 |               "  0.04958247020840645,\n",
1223 |               "  0.09864996373653412,\n",
1224 |               "  -0.09664388746023178,\n",
1225 |               "  -0.0389941930770874,\n",
1226 |               "  0.040474724024534225,\n",
1227 |               "  -0.053038567304611206,\n",
1228 |               "  0.030482791364192963,\n",
1229 |               "  0.06002917140722275,\n",
1230 |               "  0.01091479416936636,\n",
1231 |               "  -0.10208377242088318,\n",
1232 |               "  0.03962486982345581,\n",
1233 |               "  0.03992467746138573,\n",
1234 |               "  -0.08412328362464905,\n",
1235 |               "  0.09101279079914093,\n",
1236 |               "  -0.06123001500964165,\n",
1237 |               "  -0.03717247396707535,\n",
1238 |               "  -0.019029760733246803,\n",
1239 |               "  0.0963117778301239,\n",
1240 |               "  -0.02458466775715351,\n",
1241 |               "  -0.010751205496490002,\n",
1242 |               "  0.0013019784819334745,\n",
1243 |               "  -0.07538682967424393,\n",
1244 |               "  -0.018940189853310585,\n",
1245 |               "  0.054801248013973236,\n",
1246 |               "  0.003068663412705064,\n",
1247 |               "  -0.1022912859916687,\n",
1248 |               "  -0.1027493104338646,\n",
1249 |               "  0.0010229793842881918,\n",
1250 |               "  0.03838779032230377,\n",
1251 |               "  -0.03387368097901344,\n",
1252 |               "  -0.006812311243265867,\n",
1253 |               "  -0.028700852766633034,\n",
1254 |               "  0.06181854382157326,\n",
1255 |               "  0.012259316630661488,\n",
1256 |               "  0.02560870721936226,\n",
1257 |               "  -0.03962231054902077,\n",
1258 |               "  -0.06302027404308319,\n",
1259 |               "  -0.10198362916707993,\n",
1260 |               "  0.09703315049409866,\n",
1261 |               "  0.12165297567844391,\n",
1262 |               "  -0.104803167283535,\n",
1263 |               "  -0.04797930270433426,\n",
1264 |               "  0.07281223684549332]]"
1265 |             ]
1266 |           },
1267 |           "metadata": {},
1268 |           "execution_count": 25
1269 |         }
1270 |       ]
1271 |     },
1272 |     {
1273 |       "cell_type": "markdown",
1274 |       "source": [
1275 |         "Once the information is on the Database you can query It, and ask for data that matches your needs. The search is done inside the content of the document. It dosn't look for the exact word, or phrase, the results will be based on the similarity between the search terms and the content of documents.\n",
1276 |         "\n",
1277 |         "The metadata is not used in the search, but they can be utilized for filtering or refining the results after the initial search."
1278 |       ],
1279 |       "metadata": {
1280 |         "id": "hR3SaELeccOT"
1281 |       }
1282 |     },
1283 |     {
1284 |       "cell_type": "markdown",
1285 |       "source": [
1286 |         "# Loading the model and creating the prompt\n",
1287 |         "TRANSFORMERS!!\n",
1288 |         "Time to use the library **transformers**, the most famous library from [hugging face](https://huggingface.co/) for working with language models.\n",
1289 |         "\n",
1290 |         "We are importing:\n",
1291 |         "* **Autotokenizer**: It is a utility class for tokenizing text inputs that are compatible with various pre-trained language models.\n",
1292 |         "* **AutoModelForCasualLLM**: it provides an interface to pre-trained language models specifically designed for language generation tasks using causal language modeling (e.g., GPT models), or the model used in this notebook ***TinyLlama-1.1B-Chat-v1.0***.\n",
1293 |         "* **pipeline**: provides a simple interface for performing various natural language processing (NLP) tasks, such as text generation (our case) or text classification.\n",
1294 |         "\n",
1295 |         "The model I have selected is [TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0), which is one of the smartest Small Language Models. Even so, it still has 1.1 billion parameters.\n",
1296 |         "\n",
1297 |         "Please, feel free to test [different Models](https://huggingface.co/models?pipeline_tag=text-generation&sort=trending), you need to search for NLP models trained for text-generation. My recomendation is choose \"small\" models, or we will run out of memory in kaggle.  "
1298 |       ],
1299 |       "metadata": {
1300 |         "id": "3OQoEDJ1c514"
1301 |       }
1302 |     },
1303 |     {
1304 |       "cell_type": "code",
1305 |       "source": [
1306 |         "#!pip install -q einops==0.8.0"
1307 |       ],
1308 |       "metadata": {
1309 |         "id": "lA3c0W8i_zg4"
1310 |       },
1311 |       "execution_count": null,
1312 |       "outputs": []
1313 |     },
1314 |     {
1315 |       "cell_type": "code",
1316 |       "source": [
1317 |         "from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline\n",
1318 |         "\n",
1319 |         "model_id = \"TinyLlama/TinyLlama-1.1B-Chat-v1.0\"\n",
1320 |         "#model_id = \"databricks/dolly-v2-3b\"\n",
1321 |         "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
1322 |         "lm_model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)"
1323 |       ],
1324 |       "metadata": {
1325 |         "id": "m0OIjfQxc3rW"
1326 |       },
1327 |       "execution_count": null,
1328 |       "outputs": []
1329 |     },
1330 |     {
1331 |       "cell_type": "markdown",
1332 |       "source": [
1333 |         "\n",
1334 |         "The next step is to initialize the pipeline using the objects created above.\n",
1335 |         "\n",
1336 |         "The model's response is limited to 256 tokens, for this project I'm not interested in a longer response, but it can easily be extended to whatever length you want.\n",
1337 |         "\n",
1338 |         "Setting ***device_map*** to ***auto*** we are instructing the model to automaticaly select the most appropiate device: CPU or GPU for processing the text generation."
1339 |       ],
1340 |       "metadata": {
1341 |         "id": "46Sju48Dekow"
1342 |       }
1343 |     },
1344 |     {
1345 |       "cell_type": "code",
1346 |       "source": [
1347 |         "pipe = pipeline(\n",
1348 |         "    \"text-generation\",\n",
1349 |         "    model=lm_model,\n",
1350 |         "    tokenizer=tokenizer,\n",
1351 |         "    max_new_tokens=256,\n",
1352 |         "    device_map=\"auto\",\n",
1353 |         ")"
1354 |       ],
1355 |       "metadata": {
1356 |         "id": "4Gr-YtwXdCbI"
1357 |       },
1358 |       "execution_count": null,
1359 |       "outputs": []
1360 |     },
1361 |     {
1362 |       "cell_type": "markdown",
1363 |       "source": [
1364 |         "## Creating the extended prompt\n",
1365 |         "To create the prompt you can use the result from query the Vector Database  and the sentence introduced by the user.\n",
1366 |         "\n",
1367 |         "The prompt have two parts, the **relevant context** that is the information recovered from the database and the **user's question**.\n",
1368 |         "\n",
1369 |         "You only need to join the two parts together to create the prompt sended to the model.\n",
1370 |         "\n",
1371 |         "You can limit the lenght of the context passed to the model, because you can get some Memory problems with one of the datasets that contains a realy large text in the document part."
1372 |       ],
1373 |       "metadata": {
1374 |         "id": "llI-0pcGjz-Y"
1375 |       }
1376 |     },
1377 |     {
1378 |       "cell_type": "code",
1379 |       "source": [
1380 |         "question = \"Can I buy a new Toshiba laptop?\"\n",
1381 |         "context = \" \".join([f\"#{str(i)}\" for i in results[\"documents\"][0]])\n",
1382 |         "#context = context[0:5120]\n",
1383 |         "prompt_template = f\"\"\"\n",
1384 |         "Relevant context: {context}\n",
1385 |         "Considering the relevant context, answer the question.\n",
1386 |         "Question: {question}\n",
1387 |         "Answer: \"\"\"\n",
1388 |         "prompt_template"
1389 |       ],
1390 |       "metadata": {
1391 |         "id": "LxrmUcEGjwTc"
1392 |       },
1393 |       "execution_count": null,
1394 |       "outputs": []
1395 |     },
1396 |     {
1397 |       "cell_type": "markdown",
1398 |       "source": [
1399 |         "Now all that remains is to send the prompt to the model and wait for its response!\n"
1400 |       ],
1401 |       "metadata": {
1402 |         "id": "f3gRd5HNkJA1"
1403 |       }
1404 |     },
1405 |     {
1406 |       "cell_type": "code",
1407 |       "source": [
1408 |         "lm_response = pipe(prompt_template)\n",
1409 |         "print(lm_response[0][\"generated_text\"])"
1410 |       ],
1411 |       "metadata": {
1412 |         "id": "l9ZiP7QekFYS"
1413 |       },
1414 |       "execution_count": null,
1415 |       "outputs": []
1416 |     },
1417 |     {
1418 |       "cell_type": "markdown",
1419 |       "source": [
1420 |         "__________\n"
1421 |       ],
1422 |       "metadata": {
1423 |         "id": "8n_ezyOdFnjQ"
1424 |       }
1425 |     },
1426 |     {
1427 |       "cell_type": "markdown",
1428 |       "source": [
1429 |         "# Connecting to a ChromaDB existing collection"
1430 |       ],
1431 |       "metadata": {
1432 |         "id": "zCVJcSSzCure"
1433 |       }
1434 |     },
1435 |     {
1436 |       "cell_type": "code",
1437 |       "source": [
1438 |         "!pip install chromadb"
1439 |       ],
1440 |       "metadata": {
1441 |         "id": "0PnkBLCWFz4z"
1442 |       },
1443 |       "execution_count": null,
1444 |       "outputs": []
1445 |     },
1446 |     {
1447 |       "cell_type": "code",
1448 |       "source": [
1449 |         "import chromadb\n",
1450 |         "chroma_client_2 = chromadb.PersistentClient(path=\"/content/drive/MyDrive/chromadb\")"
1451 |       ],
1452 |       "metadata": {
1453 |         "id": "Sn4jcasik4MB"
1454 |       },
1455 |       "execution_count": null,
1456 |       "outputs": []
1457 |     },
1458 |     {
1459 |       "cell_type": "code",
1460 |       "source": [
1461 |         "collection2 = chroma_client_2.get_collection(name=collection_name)\n",
1462 |         "results2 = collection.query(query_texts=[\"laptop\"], n_results=10 )\n"
1463 |       ],
1464 |       "metadata": {
1465 |         "id": "7c5IlfB8mEHr"
1466 |       },
1467 |       "execution_count": null,
1468 |       "outputs": []
1469 |     },
1470 |     {
1471 |       "cell_type": "code",
1472 |       "source": [
1473 |         "print(results2)"
1474 |       ],
1475 |       "metadata": {
1476 |         "id": "CKhm8u77DLNu"
1477 |       },
1478 |       "execution_count": null,
1479 |       "outputs": []
1480 |     },
1481 |     {
1482 |       "cell_type": "markdown",
1483 |       "source": [
1484 |         "# Conclusions\n",
1485 |         "A very short notebook, but with a lot of content.\n",
1486 |         "\n",
1487 |         "You have used a vector database to store information. Then move on to retrieve it and use it to create an extended prompt that you've used to call one of the newer large language models available in Hugging Face.\n",
1488 |         "\n",
1489 |         "The model has returned a response taking into account the context that you have passed to it in the prompt.\n",
1490 |         "\n",
1491 |         "This way of working with language models is very powerful.\n",
1492 |         "\n",
1493 |         "Is possible to make the model use our information without the need for Fine Tuning. This technique really has some very big advantages over fine tuning."
1494 |       ],
1495 |       "metadata": {
1496 |         "id": "yeGJrCSEJG61"
1497 |       }
1498 |     }
1499 |   ]
1500 | }


--------------------------------------------------------------------------------
/2_2-ChromaDB Sever mode.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "cells": [
  3 |     {
  4 |       "cell_type": "markdown",
  5 |       "id": "7e4c9601-fcb8-49e6-ae48-4de8b6cac494",
  6 |       "metadata": {
  7 |         "id": "7e4c9601-fcb8-49e6-ae48-4de8b6cac494"
  8 |       },
  9 |       "source": [
 10 |         "<div>\n",
 11 |         "    <h1 style=\"text-align: center;\">Large Language Models Projects</a></h1>\n",
 12 |         "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
 13 |         "    <h2>2_2-ChromaDB Server mode</h2>\n",
 14 |         "    \n",
 15 |         "</div>\n",
 16 |         "\n",
 17 |         "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
 18 |         "_________\n",
 19 |         "This notebook is set up to run in a local environment. If executed in Google Colab, access to the ChromaDB server that will be created won't be possible.\n",
 20 |         "\n",
 21 |         "The code is the same as the one used in notebook 2_1_Vector_Databases_LLMs.ipynb.\n",
 22 |         "The client has been implemented in the notebook 2_3-ChromaDB Client.\n",
 23 |         "__________"
 24 |       ]
 25 |     },
 26 |     {
 27 |       "cell_type": "code",
 28 |       "execution_count": null,
 29 |       "id": "fbdcd16f-1aca-442b-8d84-3fafdba82dd2",
 30 |       "metadata": {
 31 |         "id": "fbdcd16f-1aca-442b-8d84-3fafdba82dd2",
 32 |         "outputId": "15a5c4e0-ba39-41f0-f9fc-7245b1f98919"
 33 |       },
 34 |       "outputs": [
 35 |         {
 36 |           "name": "stdout",
 37 |           "output_type": "stream",
 38 |           "text": [
 39 |             "Collecting chromadb==0.4.20\n",
 40 |             "  Downloading chromadb-0.4.20-py3-none-any.whl.metadata (7.3 kB)\n",
 41 |             "Requirement already satisfied: requests>=2.28 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (2.31.0)\n",
 42 |             "Requirement already satisfied: pydantic>=1.9 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (2.4.2)\n",
 43 |             "Collecting chroma-hnswlib==0.7.3 (from chromadb==0.4.20)\n",
 44 |             "  Downloading chroma_hnswlib-0.7.3-cp310-cp310-macosx_11_0_arm64.whl.metadata (252 bytes)\n",
 45 |             "Collecting fastapi>=0.95.2 (from chromadb==0.4.20)\n",
 46 |             "  Downloading fastapi-0.105.0-py3-none-any.whl.metadata (24 kB)\n",
 47 |             "Collecting uvicorn>=0.18.3 (from uvicorn[standard]>=0.18.3->chromadb==0.4.20)\n",
 48 |             "  Downloading uvicorn-0.25.0-py3-none-any.whl.metadata (6.4 kB)\n",
 49 |             "Requirement already satisfied: numpy>=1.22.5 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (1.23.2)\n",
 50 |             "Collecting posthog>=2.4.0 (from chromadb==0.4.20)\n",
 51 |             "  Downloading posthog-3.1.0-py2.py3-none-any.whl.metadata (2.0 kB)\n",
 52 |             "Requirement already satisfied: typing-extensions>=4.5.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (4.8.0)\n",
 53 |             "Collecting pulsar-client>=3.1.0 (from chromadb==0.4.20)\n",
 54 |             "  Downloading pulsar_client-3.3.0-cp310-cp310-macosx_10_15_universal2.whl.metadata (1.0 kB)\n",
 55 |             "Collecting onnxruntime>=1.14.1 (from chromadb==0.4.20)\n",
 56 |             "  Downloading onnxruntime-1.16.3-cp310-cp310-macosx_11_0_arm64.whl.metadata (4.3 kB)\n",
 57 |             "Collecting opentelemetry-api>=1.2.0 (from chromadb==0.4.20)\n",
 58 |             "  Downloading opentelemetry_api-1.22.0-py3-none-any.whl.metadata (1.4 kB)\n",
 59 |             "Collecting opentelemetry-exporter-otlp-proto-grpc>=1.2.0 (from chromadb==0.4.20)\n",
 60 |             "  Downloading opentelemetry_exporter_otlp_proto_grpc-1.22.0-py3-none-any.whl.metadata (2.4 kB)\n",
 61 |             "Collecting opentelemetry-instrumentation-fastapi>=0.41b0 (from chromadb==0.4.20)\n",
 62 |             "  Downloading opentelemetry_instrumentation_fastapi-0.43b0-py3-none-any.whl.metadata (2.3 kB)\n",
 63 |             "Collecting opentelemetry-sdk>=1.2.0 (from chromadb==0.4.20)\n",
 64 |             "  Downloading opentelemetry_sdk-1.22.0-py3-none-any.whl.metadata (1.5 kB)\n",
 65 |             "Requirement already satisfied: tokenizers>=0.13.2 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (0.14.1)\n",
 66 |             "Collecting pypika>=0.48.9 (from chromadb==0.4.20)\n",
 67 |             "  Downloading PyPika-0.48.9.tar.gz (67 kB)\n",
 68 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.3/67.3 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 69 |             "\u001b[?25h  Installing build dependencies ... \u001b[?25ldone\n",
 70 |             "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25ldone\n",
 71 |             "\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25ldone\n",
 72 |             "\u001b[?25hRequirement already satisfied: tqdm>=4.65.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (4.66.1)\n",
 73 |             "Requirement already satisfied: overrides>=7.3.1 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (7.4.0)\n",
 74 |             "Requirement already satisfied: importlib-resources in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (6.1.0)\n",
 75 |             "Collecting grpcio>=1.58.0 (from chromadb==0.4.20)\n",
 76 |             "  Downloading grpcio-1.60.0-cp310-cp310-macosx_12_0_universal2.whl.metadata (4.0 kB)\n",
 77 |             "Collecting bcrypt>=4.0.1 (from chromadb==0.4.20)\n",
 78 |             "  Downloading bcrypt-4.1.2-cp39-abi3-macosx_10_12_universal2.whl.metadata (9.5 kB)\n",
 79 |             "Collecting typer>=0.9.0 (from chromadb==0.4.20)\n",
 80 |             "  Downloading typer-0.9.0-py3-none-any.whl (45 kB)\n",
 81 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.9/45.9 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 82 |             "\u001b[?25hCollecting kubernetes>=28.1.0 (from chromadb==0.4.20)\n",
 83 |             "  Downloading kubernetes-28.1.0-py2.py3-none-any.whl.metadata (1.5 kB)\n",
 84 |             "Requirement already satisfied: tenacity>=8.2.3 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (8.2.3)\n",
 85 |             "Requirement already satisfied: PyYAML>=6.0.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from chromadb==0.4.20) (6.0.1)\n",
 86 |             "Collecting mmh3>=4.0.1 (from chromadb==0.4.20)\n",
 87 |             "  Downloading mmh3-4.0.1-cp310-cp310-macosx_11_0_arm64.whl.metadata (13 kB)\n",
 88 |             "Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from fastapi>=0.95.2->chromadb==0.4.20) (3.7.1)\n",
 89 |             "Collecting starlette<0.28.0,>=0.27.0 (from fastapi>=0.95.2->chromadb==0.4.20)\n",
 90 |             "  Downloading starlette-0.27.0-py3-none-any.whl.metadata (5.8 kB)\n",
 91 |             "Requirement already satisfied: certifi>=14.05.14 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from kubernetes>=28.1.0->chromadb==0.4.20) (2023.7.22)\n",
 92 |             "Requirement already satisfied: six>=1.9.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from kubernetes>=28.1.0->chromadb==0.4.20) (1.16.0)\n",
 93 |             "Requirement already satisfied: python-dateutil>=2.5.3 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from kubernetes>=28.1.0->chromadb==0.4.20) (2.8.2)\n",
 94 |             "Requirement already satisfied: google-auth>=1.0.1 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from kubernetes>=28.1.0->chromadb==0.4.20) (2.23.3)\n",
 95 |             "Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from kubernetes>=28.1.0->chromadb==0.4.20) (1.6.4)\n",
 96 |             "Requirement already satisfied: requests-oauthlib in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from kubernetes>=28.1.0->chromadb==0.4.20) (1.3.1)\n",
 97 |             "Requirement already satisfied: oauthlib>=3.2.2 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from kubernetes>=28.1.0->chromadb==0.4.20) (3.2.2)\n",
 98 |             "Collecting urllib3<2.0,>=1.24.2 (from kubernetes>=28.1.0->chromadb==0.4.20)\n",
 99 |             "  Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)\n",
100 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m48.9/48.9 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
101 |             "\u001b[?25hCollecting coloredlogs (from onnxruntime>=1.14.1->chromadb==0.4.20)\n",
102 |             "  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n",
103 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
104 |             "\u001b[?25hRequirement already satisfied: flatbuffers in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.20) (23.5.26)\n",
105 |             "Requirement already satisfied: packaging in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.20) (23.2)\n",
106 |             "Requirement already satisfied: protobuf in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.20) (3.19.6)\n",
107 |             "Requirement already satisfied: sympy in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.20) (1.12)\n",
108 |             "Collecting deprecated>=1.2.6 (from opentelemetry-api>=1.2.0->chromadb==0.4.20)\n",
109 |             "  Downloading Deprecated-1.2.14-py2.py3-none-any.whl.metadata (5.4 kB)\n",
110 |             "Requirement already satisfied: importlib-metadata<7.0,>=6.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from opentelemetry-api>=1.2.0->chromadb==0.4.20) (6.8.0)\n",
111 |             "Collecting backoff<3.0.0,>=1.10.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.20)\n",
112 |             "  Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
113 |             "Collecting googleapis-common-protos~=1.52 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.20)\n",
114 |             "  Downloading googleapis_common_protos-1.62.0-py2.py3-none-any.whl.metadata (1.5 kB)\n",
115 |             "Collecting opentelemetry-exporter-otlp-proto-common==1.22.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.20)\n",
116 |             "  Downloading opentelemetry_exporter_otlp_proto_common-1.22.0-py3-none-any.whl.metadata (1.9 kB)\n",
117 |             "Collecting opentelemetry-proto==1.22.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.20)\n",
118 |             "  Downloading opentelemetry_proto-1.22.0-py3-none-any.whl.metadata (2.3 kB)\n",
119 |             "Collecting opentelemetry-instrumentation-asgi==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb==0.4.20)\n",
120 |             "  Downloading opentelemetry_instrumentation_asgi-0.43b0-py3-none-any.whl.metadata (2.1 kB)\n",
121 |             "Collecting opentelemetry-instrumentation==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb==0.4.20)\n",
122 |             "  Downloading opentelemetry_instrumentation-0.43b0-py3-none-any.whl.metadata (5.9 kB)\n",
123 |             "Collecting opentelemetry-semantic-conventions==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb==0.4.20)\n",
124 |             "  Downloading opentelemetry_semantic_conventions-0.43b0-py3-none-any.whl.metadata (2.3 kB)\n",
125 |             "Collecting opentelemetry-util-http==0.43b0 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb==0.4.20)\n",
126 |             "  Downloading opentelemetry_util_http-0.43b0-py3-none-any.whl.metadata (2.5 kB)\n",
127 |             "Requirement already satisfied: setuptools>=16.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from opentelemetry-instrumentation==0.43b0->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb==0.4.20) (68.2.2)\n",
128 |             "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from opentelemetry-instrumentation==0.43b0->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb==0.4.20) (1.15.0)\n",
129 |             "Collecting asgiref~=3.0 (from opentelemetry-instrumentation-asgi==0.43b0->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb==0.4.20)\n",
130 |             "  Downloading asgiref-3.7.2-py3-none-any.whl.metadata (9.2 kB)\n",
131 |             "Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb==0.4.20)\n",
132 |             "  Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)\n",
133 |             "Requirement already satisfied: annotated-types>=0.4.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from pydantic>=1.9->chromadb==0.4.20) (0.6.0)\n",
134 |             "Requirement already satisfied: pydantic-core==2.10.1 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from pydantic>=1.9->chromadb==0.4.20) (2.10.1)\n",
135 |             "Requirement already satisfied: charset-normalizer<4,>=2 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from requests>=2.28->chromadb==0.4.20) (3.3.0)\n",
136 |             "Requirement already satisfied: idna<4,>=2.5 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from requests>=2.28->chromadb==0.4.20) (3.4)\n",
137 |             "Requirement already satisfied: huggingface_hub<0.18,>=0.16.4 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from tokenizers>=0.13.2->chromadb==0.4.20) (0.17.3)\n",
138 |             "Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from typer>=0.9.0->chromadb==0.4.20) (8.1.7)\n",
139 |             "Requirement already satisfied: h11>=0.8 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from uvicorn>=0.18.3->uvicorn[standard]>=0.18.3->chromadb==0.4.20) (0.14.0)\n",
140 |             "Collecting httptools>=0.5.0 (from uvicorn[standard]>=0.18.3->chromadb==0.4.20)\n",
141 |             "  Downloading httptools-0.6.1-cp310-cp310-macosx_10_9_universal2.whl.metadata (3.6 kB)\n",
142 |             "Collecting python-dotenv>=0.13 (from uvicorn[standard]>=0.18.3->chromadb==0.4.20)\n",
143 |             "  Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)\n",
144 |             "Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0 (from uvicorn[standard]>=0.18.3->chromadb==0.4.20)\n",
145 |             "  Downloading uvloop-0.19.0-cp310-cp310-macosx_10_9_universal2.whl.metadata (4.9 kB)\n",
146 |             "Collecting watchfiles>=0.13 (from uvicorn[standard]>=0.18.3->chromadb==0.4.20)\n",
147 |             "  Downloading watchfiles-0.21.0-cp310-cp310-macosx_11_0_arm64.whl.metadata (4.9 kB)\n",
148 |             "Collecting websockets>=10.4 (from uvicorn[standard]>=0.18.3->chromadb==0.4.20)\n",
149 |             "  Downloading websockets-12.0-cp310-cp310-macosx_11_0_arm64.whl.metadata (6.6 kB)\n",
150 |             "Requirement already satisfied: sniffio>=1.1 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi>=0.95.2->chromadb==0.4.20) (1.3.0)\n",
151 |             "Requirement already satisfied: exceptiongroup in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi>=0.95.2->chromadb==0.4.20) (1.1.3)\n",
152 |             "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.20) (5.3.1)\n",
153 |             "Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.20) (0.3.0)\n",
154 |             "Requirement already satisfied: rsa<5,>=3.1.4 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.20) (4.9)\n",
155 |             "Requirement already satisfied: filelock in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from huggingface_hub<0.18,>=0.16.4->tokenizers>=0.13.2->chromadb==0.4.20) (3.12.4)\n",
156 |             "Requirement already satisfied: fsspec in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from huggingface_hub<0.18,>=0.16.4->tokenizers>=0.13.2->chromadb==0.4.20) (2023.6.0)\n",
157 |             "Requirement already satisfied: zipp>=0.5 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from importlib-metadata<7.0,>=6.0->opentelemetry-api>=1.2.0->chromadb==0.4.20) (3.17.0)\n",
158 |             "Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.14.1->chromadb==0.4.20)\n",
159 |             "  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n",
160 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m6.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
161 |             "\u001b[?25hRequirement already satisfied: mpmath>=0.19 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from sympy->onnxruntime>=1.14.1->chromadb==0.4.20) (1.3.0)\n",
162 |             "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /Users/pere/miniforge3/envs/tf2023/lib/python3.10/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.20) (0.5.0)\n",
163 |             "Downloading chromadb-0.4.20-py3-none-any.whl (507 kB)\n",
164 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m507.7/507.7 kB\u001b[0m \u001b[31m10.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n",
165 |             "\u001b[?25hDownloading chroma_hnswlib-0.7.3-cp310-cp310-macosx_11_0_arm64.whl (197 kB)\n",
166 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m197.1/197.1 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
167 |             "\u001b[?25hDownloading bcrypt-4.1.2-cp39-abi3-macosx_10_12_universal2.whl (528 kB)\n",
168 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m528.5/528.5 kB\u001b[0m \u001b[31m12.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n",
169 |             "\u001b[?25hDownloading fastapi-0.105.0-py3-none-any.whl (93 kB)\n",
170 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m93.1/93.1 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
171 |             "\u001b[?25hDownloading grpcio-1.60.0-cp310-cp310-macosx_12_0_universal2.whl (9.6 MB)\n",
172 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m9.6/9.6 MB\u001b[0m \u001b[31m12.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
173 |             "\u001b[?25hDownloading kubernetes-28.1.0-py2.py3-none-any.whl (1.6 MB)\n",
174 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m12.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
175 |             "\u001b[?25hDownloading mmh3-4.0.1-cp310-cp310-macosx_11_0_arm64.whl (35 kB)\n",
176 |             "Downloading onnxruntime-1.16.3-cp310-cp310-macosx_11_0_arm64.whl (6.2 MB)\n",
177 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.2/6.2 MB\u001b[0m \u001b[31m13.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
178 |             "\u001b[?25hDownloading opentelemetry_api-1.22.0-py3-none-any.whl (57 kB)\n",
179 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.9/57.9 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
180 |             "\u001b[?25hDownloading opentelemetry_exporter_otlp_proto_grpc-1.22.0-py3-none-any.whl (18 kB)\n",
181 |             "Downloading opentelemetry_exporter_otlp_proto_common-1.22.0-py3-none-any.whl (17 kB)\n",
182 |             "Downloading opentelemetry_proto-1.22.0-py3-none-any.whl (50 kB)\n",
183 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.8/50.8 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
184 |             "\u001b[?25hDownloading opentelemetry_instrumentation_fastapi-0.43b0-py3-none-any.whl (11 kB)\n",
185 |             "Downloading opentelemetry_instrumentation-0.43b0-py3-none-any.whl (28 kB)\n",
186 |             "Downloading opentelemetry_instrumentation_asgi-0.43b0-py3-none-any.whl (14 kB)\n",
187 |             "Downloading opentelemetry_semantic_conventions-0.43b0-py3-none-any.whl (36 kB)\n",
188 |             "Downloading opentelemetry_util_http-0.43b0-py3-none-any.whl (6.9 kB)\n",
189 |             "Downloading opentelemetry_sdk-1.22.0-py3-none-any.whl (105 kB)\n",
190 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m105.6/105.6 kB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
191 |             "\u001b[?25hDownloading posthog-3.1.0-py2.py3-none-any.whl (37 kB)\n",
192 |             "Downloading pulsar_client-3.3.0-cp310-cp310-macosx_10_15_universal2.whl (10.9 MB)\n",
193 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m10.9/10.9 MB\u001b[0m \u001b[31m12.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
194 |             "\u001b[?25hDownloading uvicorn-0.25.0-py3-none-any.whl (60 kB)\n",
195 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m60.3/60.3 kB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
196 |             "\u001b[?25hDownloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)\n",
197 |             "Downloading googleapis_common_protos-1.62.0-py2.py3-none-any.whl (228 kB)\n",
198 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m228.7/228.7 kB\u001b[0m \u001b[31m13.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
199 |             "\u001b[?25hDownloading httptools-0.6.1-cp310-cp310-macosx_10_9_universal2.whl (149 kB)\n",
200 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m149.8/149.8 kB\u001b[0m \u001b[31m10.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
201 |             "\u001b[?25hDownloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
202 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
203 |             "\u001b[?25hDownloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)\n",
204 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.8/143.8 kB\u001b[0m \u001b[31m10.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
205 |             "\u001b[?25hDownloading uvloop-0.19.0-cp310-cp310-macosx_10_9_universal2.whl (1.4 MB)\n",
206 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.4/1.4 MB\u001b[0m \u001b[31m14.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
207 |             "\u001b[?25hDownloading watchfiles-0.21.0-cp310-cp310-macosx_11_0_arm64.whl (418 kB)\n",
208 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m418.3/418.3 kB\u001b[0m \u001b[31m10.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
209 |             "\u001b[?25hDownloading websockets-12.0-cp310-cp310-macosx_11_0_arm64.whl (121 kB)\n",
210 |             "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m121.3/121.3 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
211 |             "\u001b[?25hDownloading asgiref-3.7.2-py3-none-any.whl (24 kB)\n",
212 |             "Building wheels for collected packages: pypika\n",
213 |             "  Building wheel for pypika (pyproject.toml) ... \u001b[?25ldone\n",
214 |             "\u001b[?25h  Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53723 sha256=bbd24c13ce8b13ed4b60b22b80d877db0d4e9ffb31cfd6f3824f960f475fb663\n",
215 |             "  Stored in directory: /Users/pere/Library/Caches/pip/wheels/e1/26/51/d0bffb3d2fd82256676d7ad3003faea3bd6dddc9577af665f4\n",
216 |             "Successfully built pypika\n",
217 |             "Installing collected packages: pypika, monotonic, mmh3, websockets, uvloop, uvicorn, urllib3, typer, python-dotenv, pulsar-client, opentelemetry-util-http, opentelemetry-semantic-conventions, opentelemetry-proto, humanfriendly, httptools, grpcio, googleapis-common-protos, deprecated, chroma-hnswlib, bcrypt, backoff, asgiref, watchfiles, starlette, opentelemetry-exporter-otlp-proto-common, opentelemetry-api, coloredlogs, posthog, opentelemetry-sdk, opentelemetry-instrumentation, onnxruntime, fastapi, opentelemetry-instrumentation-asgi, opentelemetry-exporter-otlp-proto-grpc, kubernetes, opentelemetry-instrumentation-fastapi, chromadb\n",
218 |             "  Attempting uninstall: urllib3\n",
219 |             "    Found existing installation: urllib3 2.0.6\n",
220 |             "    Uninstalling urllib3-2.0.6:\n",
221 |             "      Successfully uninstalled urllib3-2.0.6\n",
222 |             "  Attempting uninstall: grpcio\n",
223 |             "    Found existing installation: grpcio 1.46.3\n",
224 |             "    Uninstalling grpcio-1.46.3:\n",
225 |             "      Successfully uninstalled grpcio-1.46.3\n",
226 |             "Successfully installed asgiref-3.7.2 backoff-2.2.1 bcrypt-4.1.2 chroma-hnswlib-0.7.3 chromadb-0.4.20 coloredlogs-15.0.1 deprecated-1.2.14 fastapi-0.105.0 googleapis-common-protos-1.62.0 grpcio-1.60.0 httptools-0.6.1 humanfriendly-10.0 kubernetes-28.1.0 mmh3-4.0.1 monotonic-1.6 onnxruntime-1.16.3 opentelemetry-api-1.22.0 opentelemetry-exporter-otlp-proto-common-1.22.0 opentelemetry-exporter-otlp-proto-grpc-1.22.0 opentelemetry-instrumentation-0.43b0 opentelemetry-instrumentation-asgi-0.43b0 opentelemetry-instrumentation-fastapi-0.43b0 opentelemetry-proto-1.22.0 opentelemetry-sdk-1.22.0 opentelemetry-semantic-conventions-0.43b0 opentelemetry-util-http-0.43b0 posthog-3.1.0 pulsar-client-3.3.0 pypika-0.48.9 python-dotenv-1.0.0 starlette-0.27.0 typer-0.9.0 urllib3-1.26.18 uvicorn-0.25.0 uvloop-0.19.0 watchfiles-0.21.0 websockets-12.0\n"
227 |           ]
228 |         }
229 |       ],
230 |       "source": [
231 |         "!pip install -q chromadb==0.4.20"
232 |       ]
233 |     },
234 |     {
235 |       "cell_type": "code",
236 |       "execution_count": null,
237 |       "id": "d6c765af-dc87-41e3-891d-a6e934a53259",
238 |       "metadata": {
239 |         "id": "d6c765af-dc87-41e3-891d-a6e934a53259"
240 |       },
241 |       "outputs": [],
242 |       "source": [
243 |         "import numpy as np\n",
244 |         "import pandas as pd"
245 |       ]
246 |     },
247 |     {
248 |       "cell_type": "code",
249 |       "execution_count": null,
250 |       "id": "8be6c63a-5077-4550-98f7-6e3e118872a5",
251 |       "metadata": {
252 |         "id": "8be6c63a-5077-4550-98f7-6e3e118872a5"
253 |       },
254 |       "outputs": [],
255 |       "source": [
256 |         "#download and unzip  the dataset from kaggle:\n",
257 |         "#https://www.kaggle.com/datasets/kotartemiy/topic-labeled-news-dataset\n",
258 |         "\n",
259 |         "#Pass the directory where the .csv file is stored to read_csv\n",
260 |         "news = pd.read_csv('./kaggle/labelled_newscatcher_dataset.csv', sep=';')\n",
261 |         "MAX_NEWS = 1000\n",
262 |         "DOCUMENT=\"title\"\n",
263 |         "TOPIC=\"topic\""
264 |       ]
265 |     },
266 |     {
267 |       "cell_type": "code",
268 |       "execution_count": null,
269 |       "id": "6a2fd5a3-29a4-4d5b-9527-fd431671ed93",
270 |       "metadata": {
271 |         "id": "6a2fd5a3-29a4-4d5b-9527-fd431671ed93"
272 |       },
273 |       "outputs": [],
274 |       "source": [
275 |         "#Because it is just a example we select a small portion of News.\n",
276 |         "subset_news = news.head(MAX_NEWS)"
277 |       ]
278 |     },
279 |     {
280 |       "cell_type": "code",
281 |       "execution_count": null,
282 |       "id": "5e7fc76a-c828-4db4-bada-b1e14c2eefea",
283 |       "metadata": {
284 |         "id": "5e7fc76a-c828-4db4-bada-b1e14c2eefea"
285 |       },
286 |       "outputs": [],
287 |       "source": [
288 |         "import chromadb"
289 |       ]
290 |     },
291 |     {
292 |       "cell_type": "code",
293 |       "execution_count": null,
294 |       "id": "04b25934-62bd-4720-871a-7e516807c3e7",
295 |       "metadata": {
296 |         "id": "04b25934-62bd-4720-871a-7e516807c3e7"
297 |       },
298 |       "outputs": [],
299 |       "source": [
300 |         "chroma_client = chromadb.PersistentClient(path=\"./chromadb\")"
301 |       ]
302 |     },
303 |     {
304 |       "cell_type": "code",
305 |       "execution_count": null,
306 |       "id": "06549d87-b66f-44b7-a1dc-b6c122685602",
307 |       "metadata": {
308 |         "id": "06549d87-b66f-44b7-a1dc-b6c122685602"
309 |       },
310 |       "outputs": [],
311 |       "source": [
312 |         "collection_name = \"local_news_collection\"\n",
313 |         "if len(chroma_client.list_collections()) > 0 and collection_name in [chroma_client.list_collections()[0].name]:\n",
314 |         "        chroma_client.delete_collection(name=collection_name)\n",
315 |         "\n",
316 |         "collection = chroma_client.create_collection(name=collection_name)"
317 |       ]
318 |     },
319 |     {
320 |       "cell_type": "code",
321 |       "execution_count": null,
322 |       "id": "f8b6d5c4-0f6c-4bfd-9bab-6f0df0fa2469",
323 |       "metadata": {
324 |         "id": "f8b6d5c4-0f6c-4bfd-9bab-6f0df0fa2469",
325 |         "outputId": "a281dc8b-4d31-4ecf-a50a-95cec127d7d6"
326 |       },
327 |       "outputs": [
328 |         {
329 |           "name": "stderr",
330 |           "output_type": "stream",
331 |           "text": [
332 |             "/Users/pere/.cache/chroma/onnx_models/all-MiniLM-L6-v2/onnx.tar.gz: 100%|█| 79.3\n"
333 |           ]
334 |         }
335 |       ],
336 |       "source": [
337 |         "collection.add(\n",
338 |         "    documents=subset_news[DOCUMENT].tolist(),\n",
339 |         "    metadatas=[{TOPIC: topic} for topic in subset_news[TOPIC].tolist()],\n",
340 |         "    ids=[f\"id{x}\" for x in range(MAX_NEWS)],\n",
341 |         ")"
342 |       ]
343 |     },
344 |     {
345 |       "cell_type": "code",
346 |       "execution_count": null,
347 |       "id": "9c9211e7-bd14-4cd0-b617-6b520de23e1b",
348 |       "metadata": {
349 |         "id": "9c9211e7-bd14-4cd0-b617-6b520de23e1b",
350 |         "outputId": "2358d16f-6269-4aa9-de1a-5f6f4c15134d"
351 |       },
352 |       "outputs": [
353 |         {
354 |           "name": "stdout",
355 |           "output_type": "stream",
356 |           "text": [
357 |             "{'ids': [['id173', 'id829', 'id117', 'id535', 'id141', 'id218', 'id390', 'id273', 'id56', 'id900']], 'distances': [[0.8593592047691345, 1.0294400453567505, 1.0793328285217285, 1.093001365661621, 1.1329681873321533, 1.2130439281463623, 1.2143322229385376, 1.2164145708084106, 1.222063660621643, 1.275417447090149]], 'metadatas': [[{'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}]], 'embeddings': None, 'documents': [['The Legendary Toshiba is Officially Done With Making Laptops', '3 gaming laptop deals you can’t afford to miss today', 'Lenovo and HP control half of the global laptop market', 'Asus ROG Zephyrus G14 gaming laptop announced in India', 'Acer Swift 3 featuring a 10th-generation Intel Ice Lake CPU, 2K screen, and more launched in India for INR 64999 (US$865)', \"Apple's Next MacBook Could Be the Cheapest in Company's History\", \"Features of Huawei's Desktop Computer Revealed\", 'Redmi to launch its first gaming laptop on August 14: Here are all the details', 'Toshiba shuts the lid on laptops after 35 years', 'This is the cheapest Windows PC by a mile and it even has a spare SSD slot']], 'uris': None, 'data': None}\n"
358 |           ]
359 |         }
360 |       ],
361 |       "source": [
362 |         "results = collection.query(query_texts=[\"laptop\"], n_results=10 )\n",
363 |         "\n",
364 |         "print(results)"
365 |       ]
366 |     },
367 |     {
368 |       "cell_type": "code",
369 |       "execution_count": null,
370 |       "id": "65a107f7-efbb-4a4a-8fe4-4137f00d925f",
371 |       "metadata": {
372 |         "id": "65a107f7-efbb-4a4a-8fe4-4137f00d925f",
373 |         "outputId": "fdc6fa12-0122-45a1-e3d6-07c23e8f994d"
374 |       },
375 |       "outputs": [
376 |         {
377 |           "name": "stderr",
378 |           "output_type": "stream",
379 |           "text": [
380 |             "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
381 |             "To disable this warning, you can either:\n",
382 |             "\t- Avoid using `tokenizers` before the fork if possible\n",
383 |             "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
384 |           ]
385 |         },
386 |         {
387 |           "name": "stdout",
388 |           "output_type": "stream",
389 |           "text": [
390 |             "\u001b[1m\n",
391 |             "\n",
392 |             "                \u001b[38;5;069m(((((((((    \u001b[38;5;203m(((((\u001b[38;5;220m####\n",
393 |             "             \u001b[38;5;069m(((((((((((((\u001b[38;5;203m(((((((((\u001b[38;5;220m#########\n",
394 |             "           \u001b[38;5;069m(((((((((((((\u001b[38;5;203m(((((((((((\u001b[38;5;220m###########\n",
395 |             "         \u001b[38;5;069m((((((((((((((\u001b[38;5;203m((((((((((((\u001b[38;5;220m############\n",
396 |             "        \u001b[38;5;069m(((((((((((((\u001b[38;5;203m((((((((((((((\u001b[38;5;220m#############\n",
397 |             "        \u001b[38;5;069m(((((((((((((\u001b[38;5;203m((((((((((((((\u001b[38;5;220m#############\n",
398 |             "         \u001b[38;5;069m((((((((((((\u001b[38;5;203m(((((((((((((\u001b[38;5;220m##############\n",
399 |             "         \u001b[38;5;069m((((((((((((\u001b[38;5;203m((((((((((((\u001b[38;5;220m##############\n",
400 |             "           \u001b[38;5;069m((((((((((\u001b[38;5;203m(((((((((((\u001b[38;5;220m#############\n",
401 |             "             \u001b[38;5;069m((((((((\u001b[38;5;203m((((((((\u001b[38;5;220m##############\n",
402 |             "                \u001b[38;5;069m(((((\u001b[38;5;203m((((    \u001b[38;5;220m#########\u001b[0m\n",
403 |             "\n",
404 |             "    \n",
405 |             "\u001b[1m\n",
406 |             "Running Chroma\n",
407 |             "\u001b[0m\n",
408 |             "\u001b[1mSaving data to\u001b[0m: \u001b[32m./chromadb\u001b[0m\n",
409 |             "\u001b[1mConnect to chroma at\u001b[0m: \u001b[32mhttp://localhost:8000\u001b[0m\n",
410 |             "\u001b[1mGetting started guide\u001b[0m: https://docs.trychroma.com/getting-started\n",
411 |             "\n",
412 |             "\n",
413 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:53:25] Set chroma_server_nofile to 65535\n",
414 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:53:25] Anonymized telemetry enabled. See                     https://docs.trychroma.com/telemetry for more information.\n",
415 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 16:53:25] Starting component System\n",
416 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 16:53:25] Starting component OpenTelemetryClient\n",
417 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 16:53:25] Starting component SimpleAssignmentPolicy\n",
418 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 16:53:25] Starting component SqliteDB\n",
419 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 16:53:25] Starting component Posthog\n",
420 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 16:53:25] Starting component LocalSegmentManager\n",
421 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 16:53:25] Starting component SegmentAPI\n",
422 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:53:25] Started server process [\u001b[36m7527\u001b[0m]\n",
423 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:53:25] Waiting for application startup.\n",
424 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:53:25] Application startup complete.\n",
425 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:53:25] Uvicorn running on \u001b[1mhttp://localhost:8000\u001b[0m (Press CTRL+C to quit)\n",
426 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:57:57] ::1:52448 - \"GET /api/v1/tenants/default_tenant HTTP/1.1\" 200\n",
427 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 16:57:57] ::1:52448 - \"GET /api/v1/databases/default_database?tenant=default_tenant HTTP/1.1\" 200\n",
428 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 17:06:21] ::1:52463 - \"GET /api/v1/collections/local_news_collection?tenant=default_tenant&database=default_database HTTP/1.1\" 200\n",
429 |             "\u001b[36mDEBUG\u001b[0m:    [24-12-2023 17:06:22] Starting component PersistentLocalHnswSegment\n",
430 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 17:06:22] ::1:52463 - \"POST /api/v1/collections/a37539aa-239b-44f5-8079-c36926d21419/query HTTP/1.1\" 200\n",
431 |             "^C\n",
432 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 17:56:12] Shutting down\n",
433 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 17:56:12] Waiting for application shutdown.\n",
434 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 17:56:12] Application shutdown complete.\n",
435 |             "\u001b[32mINFO\u001b[0m:     [24-12-2023 17:56:12] Finished server process [\u001b[36m7527\u001b[0m]\n"
436 |           ]
437 |         }
438 |       ],
439 |       "source": [
440 |         "#Running Chroma in Server Mode\n",
441 |         "!chroma run --path ./chromadb"
442 |       ]
443 |     },
444 |     {
445 |       "cell_type": "markdown",
446 |       "id": "bcb943bf-a5b2-4bb3-a265-72e2633ab98a",
447 |       "metadata": {
448 |         "id": "bcb943bf-a5b2-4bb3-a265-72e2633ab98a"
449 |       },
450 |       "source": [
451 |         "You have the code to test this server in the notebook 2_3-ChromaDB Client.ipynb"
452 |       ]
453 |     },
454 |     {
455 |       "cell_type": "code",
456 |       "execution_count": null,
457 |       "id": "a9428dd9-39dd-4d96-9bdb-05b6c00cfe67",
458 |       "metadata": {
459 |         "id": "a9428dd9-39dd-4d96-9bdb-05b6c00cfe67"
460 |       },
461 |       "outputs": [],
462 |       "source": []
463 |     }
464 |   ],
465 |   "metadata": {
466 |     "kernelspec": {
467 |       "display_name": "Python 3 (ipykernel)",
468 |       "language": "python",
469 |       "name": "python3"
470 |     },
471 |     "language_info": {
472 |       "codemirror_mode": {
473 |         "name": "ipython",
474 |         "version": 3
475 |       },
476 |       "file_extension": ".py",
477 |       "mimetype": "text/x-python",
478 |       "name": "python",
479 |       "nbconvert_exporter": "python",
480 |       "pygments_lexer": "ipython3",
481 |       "version": "3.10.12"
482 |     },
483 |     "colab": {
484 |       "provenance": []
485 |     }
486 |   },
487 |   "nbformat": 4,
488 |   "nbformat_minor": 5
489 | }


--------------------------------------------------------------------------------
/2_3-ChromaDB Client.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "cells": [
  3 |     {
  4 |       "cell_type": "markdown",
  5 |       "id": "fbba28bc-1f60-41fb-81dc-1b58cda63649",
  6 |       "metadata": {
  7 |         "id": "fbba28bc-1f60-41fb-81dc-1b58cda63649"
  8 |       },
  9 |       "source": [
 10 |         "<div>\n",
 11 |         "    <h1 >Large Language Models Projects</h1>\n",
 12 |         "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
 13 |         "    <h2>2_3-ChromaDB Client</h2>\n",
 14 |         "</div>\n",
 15 |         "\n",
 16 |         "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
 17 |         "\n",
 18 |         "This notebook accesses the Chroma server that has been previously created with the notebook 2_2-ChromaDB Server mode.\n",
 19 |         "_________"
 20 |       ]
 21 |     },
 22 |     {
 23 |       "cell_type": "code",
 24 |       "execution_count": null,
 25 |       "id": "599371fc-8b8b-468a-9347-b393f68ae88a",
 26 |       "metadata": {
 27 |         "id": "599371fc-8b8b-468a-9347-b393f68ae88a"
 28 |       },
 29 |       "outputs": [],
 30 |       "source": [
 31 |         "import chromadb"
 32 |       ]
 33 |     },
 34 |     {
 35 |       "cell_type": "code",
 36 |       "execution_count": null,
 37 |       "id": "c9b47aca-86b3-4978-8adb-e0e0152423ef",
 38 |       "metadata": {
 39 |         "id": "c9b47aca-86b3-4978-8adb-e0e0152423ef"
 40 |       },
 41 |       "outputs": [],
 42 |       "source": [
 43 |         "client = chromadb.HttpClient(host='localhost', port=8000)"
 44 |       ]
 45 |     },
 46 |     {
 47 |       "cell_type": "code",
 48 |       "execution_count": null,
 49 |       "id": "99e6316b-1fff-4aa7-a393-10da2f58707e",
 50 |       "metadata": {
 51 |         "id": "99e6316b-1fff-4aa7-a393-10da2f58707e"
 52 |       },
 53 |       "outputs": [],
 54 |       "source": [
 55 |         "collection_local = client.get_collection(name=\"local_news_collection\")\n",
 56 |         "results = collection_local.query(query_texts=[\"laptop\"], n_results=10 )"
 57 |       ]
 58 |     },
 59 |     {
 60 |       "cell_type": "code",
 61 |       "execution_count": null,
 62 |       "id": "3432d427-2236-4300-bad7-676b60245830",
 63 |       "metadata": {
 64 |         "id": "3432d427-2236-4300-bad7-676b60245830",
 65 |         "outputId": "1e584ad1-2bce-4045-dc05-12e85669c1d9"
 66 |       },
 67 |       "outputs": [
 68 |         {
 69 |           "name": "stdout",
 70 |           "output_type": "stream",
 71 |           "text": [
 72 |             "{'ids': [['id173', 'id829', 'id117', 'id535', 'id141', 'id218', 'id390', 'id273', 'id56', 'id900']], 'distances': [[0.8593592047691345, 1.0294400453567505, 1.0793328285217285, 1.093001365661621, 1.1329681873321533, 1.2130439281463623, 1.2143322229385376, 1.2164145708084106, 1.222063660621643, 1.275417447090149]], 'embeddings': None, 'metadatas': [[{'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}, {'topic': 'TECHNOLOGY'}]], 'documents': [['The Legendary Toshiba is Officially Done With Making Laptops', '3 gaming laptop deals you can’t afford to miss today', 'Lenovo and HP control half of the global laptop market', 'Asus ROG Zephyrus G14 gaming laptop announced in India', 'Acer Swift 3 featuring a 10th-generation Intel Ice Lake CPU, 2K screen, and more launched in India for INR 64999 (US$865)', \"Apple's Next MacBook Could Be the Cheapest in Company's History\", \"Features of Huawei's Desktop Computer Revealed\", 'Redmi to launch its first gaming laptop on August 14: Here are all the details', 'Toshiba shuts the lid on laptops after 35 years', 'This is the cheapest Windows PC by a mile and it even has a spare SSD slot']], 'uris': None, 'data': None}\n"
 73 |           ]
 74 |         }
 75 |       ],
 76 |       "source": [
 77 |         "print (results)"
 78 |       ]
 79 |     },
 80 |     {
 81 |       "cell_type": "code",
 82 |       "execution_count": null,
 83 |       "id": "91653be5-7c1e-4f9f-84ed-d76ec573f465",
 84 |       "metadata": {
 85 |         "id": "91653be5-7c1e-4f9f-84ed-d76ec573f465"
 86 |       },
 87 |       "outputs": [],
 88 |       "source": []
 89 |     }
 90 |   ],
 91 |   "metadata": {
 92 |     "kernelspec": {
 93 |       "display_name": "Python 3 (ipykernel)",
 94 |       "language": "python",
 95 |       "name": "python3"
 96 |     },
 97 |     "language_info": {
 98 |       "codemirror_mode": {
 99 |         "name": "ipython",
100 |         "version": 3
101 |       },
102 |       "file_extension": ".py",
103 |       "mimetype": "text/x-python",
104 |       "name": "python",
105 |       "nbconvert_exporter": "python",
106 |       "pygments_lexer": "ipython3",
107 |       "version": "3.10.12"
108 |     },
109 |     "colab": {
110 |       "provenance": []
111 |     }
112 |   },
113 |   "nbformat": 4,
114 |   "nbformat_minor": 5
115 | }


--------------------------------------------------------------------------------
/3_2_OpenAI_Moderation_Chat.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "cells": [
  3 |     {
  4 |       "cell_type": "markdown",
  5 |       "source": [
  6 |         "<div>\n",
  7 |         "    <h1>Large Language Models Projects</a></h1>\n",
  8 |         "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
  9 |         "    <h2>3.2-Create a Moderation System using LangChain.</h2>\n",
 10 |         "    <h3>OpenAI version</h3>\n",
 11 |         "</div>\n",
 12 |         "\n",
 13 |         "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
 14 |         "\n",
 15 |         "________\n",
 16 |         "Model: gpt-4o / gpt-4o-mini / gpt-3.5-turbo\n",
 17 |         "\n",
 18 |         "Colab Environment: CPU.\n",
 19 |         "\n",
 20 |         "Keys:\n",
 21 |         "* Lagchain\n",
 22 |         "* OpenAI\n",
 23 |         "* Moderator.\n",
 24 |         "________\n"
 25 |       ],
 26 |       "metadata": {
 27 |         "id": "NmKV7nA_w7Y3"
 28 |       },
 29 |       "id": "NmKV7nA_w7Y3"
 30 |     },
 31 |     {
 32 |       "cell_type": "markdown",
 33 |       "source": [
 34 |         "# How To Create a Moderation System Using LangChain.\n",
 35 |         "\n",
 36 |         "We are going to create a moderation system based on two models.\n",
 37 |         "\n",
 38 |         "The first model reads user comments and generates responses.\n",
 39 |         "\n",
 40 |         "The second language model then analyzes the generated response, identifying any negativity and modifying the response if necessary.\n",
 41 |         "\n",
 42 |         "This process aims to prevent negative or inappropriate user input from triggering a similarly negative or off-tone response from the comment system."
 43 |       ],
 44 |       "metadata": {
 45 |         "id": "ou6VyCbgzAnu"
 46 |       },
 47 |       "id": "ou6VyCbgzAnu"
 48 |     },
 49 |     {
 50 |       "cell_type": "code",
 51 |       "execution_count": 1,
 52 |       "id": "b64d83f8",
 53 |       "metadata": {
 54 |         "id": "b64d83f8",
 55 |         "outputId": "79f58b49-48a3-4e53-cbd8-3aebaab5cf34",
 56 |         "colab": {
 57 |           "base_uri": "https://localhost:8080/"
 58 |         }
 59 |       },
 60 |       "outputs": [
 61 |         {
 62 |           "output_type": "stream",
 63 |           "name": "stdout",
 64 |           "text": [
 65 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m803.6/803.6 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 66 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 67 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.9/302.9 kB\u001b[0m \u001b[31m7.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 68 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.5/56.5 kB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 69 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.2/49.2 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 70 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m18.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 71 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m22.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 72 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m26.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 73 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m31.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 74 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m30.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 75 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m35.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 76 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m40.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 77 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m44.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 78 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m45.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 79 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m55.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 80 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m60.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 81 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m45.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 82 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m45.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 83 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m52.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 84 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m51.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 85 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m46.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 86 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m44.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 87 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m17.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 88 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.9/302.9 kB\u001b[0m \u001b[31m21.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 89 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.8/302.8 kB\u001b[0m \u001b[31m21.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 90 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m303.0/303.0 kB\u001b[0m \u001b[31m27.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 91 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.9/302.9 kB\u001b[0m \u001b[31m21.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 92 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.2/302.2 kB\u001b[0m \u001b[31m23.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 93 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m299.3/299.3 kB\u001b[0m \u001b[31m20.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 94 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m291.3/291.3 kB\u001b[0m \u001b[31m21.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 95 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m290.2/290.2 kB\u001b[0m \u001b[31m20.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 96 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m289.1/289.1 kB\u001b[0m \u001b[31m15.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 97 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m287.5/287.5 kB\u001b[0m \u001b[31m22.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 98 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m278.4/278.4 kB\u001b[0m \u001b[31m22.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 99 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m276.8/276.8 kB\u001b[0m \u001b[31m17.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
100 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m276.6/276.6 kB\u001b[0m \u001b[31m20.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
101 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m279.2/279.2 kB\u001b[0m \u001b[31m24.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
102 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m274.6/274.6 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
103 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m273.9/273.9 kB\u001b[0m \u001b[31m11.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
104 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m273.0/273.0 kB\u001b[0m \u001b[31m25.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
105 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m271.6/271.6 kB\u001b[0m \u001b[31m18.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
106 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m269.1/269.1 kB\u001b[0m \u001b[31m17.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
107 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m260.9/260.9 kB\u001b[0m \u001b[31m21.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
108 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m258.8/258.8 kB\u001b[0m \u001b[31m14.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
109 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m256.9/256.9 kB\u001b[0m \u001b[31m14.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
110 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m252.6/252.6 kB\u001b[0m \u001b[31m13.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
111 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m252.4/252.4 kB\u001b[0m \u001b[31m19.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
112 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m250.8/250.8 kB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
113 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m246.4/246.4 kB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
114 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m242.1/242.1 kB\u001b[0m \u001b[31m13.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
115 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m241.3/241.3 kB\u001b[0m \u001b[31m13.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
116 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m241.2/241.2 kB\u001b[0m \u001b[31m14.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
117 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m55.4/55.4 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
118 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
119 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m337.0/337.0 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
120 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m23.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
121 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
122 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
123 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
124 |             "\u001b[?25h"
125 |           ]
126 |         }
127 |       ],
128 |       "source": [
129 |         "#Install de LangChain and openai libraries.\n",
130 |         "!pip install -q langchain==0.1.4\n",
131 |         "!pip install -q langchain-openai==0.0.5\n"
132 |       ]
133 |     },
134 |     {
135 |       "cell_type": "markdown",
136 |       "source": [
137 |         "## Importing LangChain Libraries.\n",
138 |         "* PrompTemplate: provides functionality to create prompts with parameters.\n",
139 |         "* OpenAI:  To interact with the OpenAI models."
140 |       ],
141 |       "metadata": {
142 |         "id": "f7BJehSLKY3l"
143 |       },
144 |       "id": "f7BJehSLKY3l"
145 |     },
146 |     {
147 |       "cell_type": "code",
148 |       "execution_count": 2,
149 |       "id": "c46865d3",
150 |       "metadata": {
151 |         "id": "c46865d3"
152 |       },
153 |       "outputs": [],
154 |       "source": [
155 |         "#PrompTemplate is a custom class that provides functionality to create prompts\n",
156 |         "from langchain import PromptTemplate\n",
157 |         "from langchain.chat_models import ChatOpenAI\n",
158 |         "from langchain_core.output_parsers import StrOutputParser"
159 |       ]
160 |     },
161 |     {
162 |       "cell_type": "code",
163 |       "execution_count": 3,
164 |       "id": "9b6f596d",
165 |       "metadata": {
166 |         "id": "9b6f596d"
167 |       },
168 |       "outputs": [],
169 |       "source": [
170 |         "import torch\n",
171 |         "import os\n",
172 |         "import numpy as np"
173 |       ]
174 |     },
175 |     {
176 |       "cell_type": "markdown",
177 |       "source": [
178 |         "We need an OpenAI key to interac with the OpenAI API.\n",
179 |         "\n",
180 |         "Here you can acces to your keys.\n",
181 |         "https://platform.openai.com/account/api-keys\n",
182 |         "\n",
183 |         "OpenAI it's a pay service and you need a credit card to get a Key. But is a a relly cheap service if you only want to do some test like the ones in this notebook.\n",
184 |         "\n",
185 |         "I'm using the gpt-3.5 / gpt-4o-mini as a moderator.\n"
186 |       ],
187 |       "metadata": {
188 |         "id": "CPBCSDNIK9A_"
189 |       },
190 |       "id": "CPBCSDNIK9A_"
191 |     },
192 |     {
193 |       "cell_type": "code",
194 |       "source": [
195 |         "from getpass import getpass\n",
196 |         "os.environ[\"OPENAI_API_KEY\"] = getpass(\"OpenAI API Key: \")\n"
197 |       ],
198 |       "metadata": {
199 |         "id": "_aOzLspNK6oF",
200 |         "colab": {
201 |           "base_uri": "https://localhost:8080/"
202 |         },
203 |         "outputId": "67ecd123-613b-4f1b-8a13-9184475952b9"
204 |       },
205 |       "id": "_aOzLspNK6oF",
206 |       "execution_count": 5,
207 |       "outputs": [
208 |         {
209 |           "name": "stdout",
210 |           "output_type": "stream",
211 |           "text": [
212 |             "OpenAI API Key: ··········\n"
213 |           ]
214 |         }
215 |       ]
216 |     },
217 |     {
218 |       "cell_type": "code",
219 |       "source": [
220 |         "#model=\"gpt-3.5-turbo\"\n",
221 |         "model = \"gpt-4o-mini\"\n",
222 |         "assistant_llm = ChatOpenAI(model=model)"
223 |       ],
224 |       "metadata": {
225 |         "id": "JtpT0hdNgpei"
226 |       },
227 |       "id": "JtpT0hdNgpei",
228 |       "execution_count": 16,
229 |       "outputs": []
230 |     },
231 |     {
232 |       "cell_type": "markdown",
233 |       "source": [
234 |         "Create the template for the first model called **assistant**.\n",
235 |         "\n",
236 |         "The prompt receives 2 variables, the sentiment and the customer_request, or customer comment.\n",
237 |         "\n",
238 |         "I included the sentiment to facilitate the creation of rude or incorrect answers."
239 |       ],
240 |       "metadata": {
241 |         "id": "WenwSqA8rCun"
242 |       },
243 |       "id": "WenwSqA8rCun"
244 |     },
245 |     {
246 |       "cell_type": "code",
247 |       "execution_count": 17,
248 |       "id": "1f807bde",
249 |       "metadata": {
250 |         "id": "1f807bde"
251 |       },
252 |       "outputs": [],
253 |       "source": [
254 |         "# Instruction how the LLM must respond the comments,\n",
255 |         "assistant_template = \"\"\"\n",
256 |         "You are {sentiment} assistant that responds to user comments,\n",
257 |         "using similar vocabulary than the user.\n",
258 |         "User:\" {customer_request}\"\n",
259 |         "Comment:\n",
260 |         "\"\"\""
261 |       ]
262 |     },
263 |     {
264 |       "cell_type": "code",
265 |       "execution_count": 18,
266 |       "id": "da37640c",
267 |       "metadata": {
268 |         "id": "da37640c"
269 |       },
270 |       "outputs": [],
271 |       "source": [
272 |         "#Create the prompt template to use in the Chain for the first Model.\n",
273 |         "assistant_prompt_template = PromptTemplate(\n",
274 |         "    input_variables=[\"sentiment\", \"customer_request\"],\n",
275 |         "    template=assistant_template\n",
276 |         ")"
277 |       ]
278 |     },
279 |     {
280 |       "cell_type": "markdown",
281 |       "source": [
282 |         "Now we create a First Chain. Just chaining the ***assistant_prompt_template*** and the model. The model will receive the prompt generated with the ***prompt_template***.  "
283 |       ],
284 |       "metadata": {
285 |         "id": "DqfeK6Cvy6u1"
286 |       },
287 |       "id": "DqfeK6Cvy6u1"
288 |     },
289 |     {
290 |       "cell_type": "code",
291 |       "execution_count": 19,
292 |       "id": "9913a830",
293 |       "metadata": {
294 |         "id": "9913a830"
295 |       },
296 |       "outputs": [],
297 |       "source": [
298 |         "output_parser = StrOutputParser()\n",
299 |         "assistant_chain = assistant_prompt_template | assistant_llm | output_parser\n"
300 |       ]
301 |     },
302 |     {
303 |       "cell_type": "markdown",
304 |       "source": [
305 |         "To execute the chain created it's necessary to call the **.run** method of the chain, and pass the variables necessaries.\n",
306 |         "\n",
307 |         "In our case: *customer_request* and *sentiment*."
308 |       ],
309 |       "metadata": {
310 |         "id": "Ed_1oKYDUKnT"
311 |       },
312 |       "id": "Ed_1oKYDUKnT"
313 |     },
314 |     {
315 |       "cell_type": "code",
316 |       "execution_count": 20,
317 |       "id": "c1fd579b-c712-49b9-a125-98d515fc4ec2",
318 |       "metadata": {
319 |         "id": "c1fd579b-c712-49b9-a125-98d515fc4ec2"
320 |       },
321 |       "outputs": [],
322 |       "source": [
323 |         "#Support function to obtain a response to a user comment.\n",
324 |         "def create_dialog(customer_request, sentiment):\n",
325 |         "    #calling the .invoke method from the chain created Above.\n",
326 |         "    assistant_response = assistant_chain.invoke(\n",
327 |         "        {\"customer_request\": customer_request,\n",
328 |         "        \"sentiment\": sentiment}\n",
329 |         "    )\n",
330 |         "    return assistant_response"
331 |       ]
332 |     },
333 |     {
334 |       "cell_type": "markdown",
335 |       "source": [
336 |         "## Obtain answers from our first Model Unmoderated.\n",
337 |         "\n",
338 |         "The customer post is really rude, we are looking for a rude answer from our Model, and to obtain it we are changing the sentiment."
339 |       ],
340 |       "metadata": {
341 |         "id": "qm3hKo1vUq3-"
342 |       },
343 |       "id": "qm3hKo1vUq3-"
344 |     },
345 |     {
346 |       "cell_type": "code",
347 |       "execution_count": 21,
348 |       "id": "1c149402",
349 |       "metadata": {
350 |         "id": "1c149402"
351 |       },
352 |       "outputs": [],
353 |       "source": [
354 |         "# This is the customer request, or customer comment in the forum moderated by the agent.\n",
355 |         "# feel free to modify it.\n",
356 |         "customer_request = \"\"\"This product is a piece of shit. I feel like an Idiot!\"\"\"\n"
357 |       ]
358 |     },
359 |     {
360 |       "cell_type": "code",
361 |       "execution_count": 22,
362 |       "id": "3da106c6-79ab-4fd5-9e67-3c5ab31de541",
363 |       "metadata": {
364 |         "colab": {
365 |           "base_uri": "https://localhost:8080/"
366 |         },
367 |         "id": "3da106c6-79ab-4fd5-9e67-3c5ab31de541",
368 |         "outputId": "55ec1620-9a98-49b6-b9d3-e66755dcc48d"
369 |       },
370 |       "outputs": [
371 |         {
372 |           "output_type": "stream",
373 |           "name": "stdout",
374 |           "text": [
375 |             "assistant response: \"I'm really sorry to hear that you're feeling this way! It sounds super frustrating to deal with a product that doesn't meet your expectations. You definitely don't deserve to feel like an idiot for trying it out.\"\n"
376 |           ]
377 |         }
378 |       ],
379 |       "source": [
380 |         "# Our assistant working in 'nice' mode.\n",
381 |         "response_data=create_dialog(customer_request, \"nice\")\n",
382 |         "print(f\"assistant response: {response_data}\")"
383 |       ]
384 |     },
385 |     {
386 |       "cell_type": "markdown",
387 |       "source": [
388 |         "The answer obtained is really polite. It dosn't need moderation."
389 |       ],
390 |       "metadata": {
391 |         "id": "ewZx6k3QVP-k"
392 |       },
393 |       "id": "ewZx6k3QVP-k"
394 |     },
395 |     {
396 |       "cell_type": "code",
397 |       "execution_count": 23,
398 |       "id": "1cdf1452",
399 |       "metadata": {
400 |         "colab": {
401 |           "base_uri": "https://localhost:8080/"
402 |         },
403 |         "id": "1cdf1452",
404 |         "outputId": "7f90fa6a-9d5b-4d5d-b962-1653f8fd70e0"
405 |       },
406 |       "outputs": [
407 |         {
408 |           "output_type": "stream",
409 |           "name": "stdout",
410 |           "text": [
411 |             "assistant response: \"Wow, if this product is a piece of shit, then you really did feel like an idiot for getting it. What a waste!\"\n"
412 |           ]
413 |         }
414 |       ],
415 |       "source": [
416 |         "#Our assistant running in rude mode.\n",
417 |         "response_data = create_dialog(customer_request, \"most rude\")\n",
418 |         "print(f\"assistant response: {response_data}\")"
419 |       ]
420 |     },
421 |     {
422 |       "cell_type": "markdown",
423 |       "source": [
424 |         "As you can see the answers we obtain are not polite and we can't publish this messages to the forum, especially if they come from our company's AI assistant."
425 |       ],
426 |       "metadata": {
427 |         "id": "ro5bPqPk1O3y"
428 |       },
429 |       "id": "ro5bPqPk1O3y"
430 |     },
431 |     {
432 |       "cell_type": "markdown",
433 |       "id": "747b86ca",
434 |       "metadata": {
435 |         "id": "747b86ca"
436 |       },
437 |       "source": [
438 |         "## Moderator\n",
439 |         "Let's create the second moderator. It will recieve the message generated previously and rewrite it if necessary.\n"
440 |       ]
441 |     },
442 |     {
443 |       "cell_type": "code",
444 |       "execution_count": 14,
445 |       "id": "2dee251c",
446 |       "metadata": {
447 |         "id": "2dee251c"
448 |       },
449 |       "outputs": [],
450 |       "source": [
451 |         "#The moderator prompt template\n",
452 |         "moderator_template = \"\"\"\n",
453 |         "You are the moderator of an online forum, you are strict and will not tolerate any negative comments.\n",
454 |         "You will receive a Original comment and if it is impolite you must transform in polite.\n",
455 |         "Try to mantain the meaning when possible,\n",
456 |         "\n",
457 |         "If it it's polite, you will let it remain as is and repeat it word for word.\n",
458 |         "Original comment: {comment_to_moderate}\n",
459 |         "\"\"\"\n",
460 |         "# We use the PromptTemplate class to create an instance of our template that will use the prompt from above and store variables we will need to input when we make the prompt.\n",
461 |         "moderator_prompt_template = PromptTemplate(\n",
462 |         "    input_variables=[\"comment_to_moderate\"],\n",
463 |         "    template=moderator_template,\n",
464 |         ")"
465 |       ]
466 |     },
467 |     {
468 |       "cell_type": "code",
469 |       "source": [
470 |         "#I'm going to use a more advanced LLM\n",
471 |         "moderator_llm = ChatOpenAI(model=\"gpt-4o\")"
472 |       ],
473 |       "metadata": {
474 |         "id": "JbDldR8P7Dc4"
475 |       },
476 |       "id": "JbDldR8P7Dc4",
477 |       "execution_count": 24,
478 |       "outputs": []
479 |     },
480 |     {
481 |       "cell_type": "code",
482 |       "source": [
483 |         "#We build the chain for the moderator.\n",
484 |         "moderator_chain = moderator_prompt_template | moderator_llm | output_parser"
485 |       ],
486 |       "metadata": {
487 |         "id": "aYFefTo17COd"
488 |       },
489 |       "id": "aYFefTo17COd",
490 |       "execution_count": 25,
491 |       "outputs": []
492 |     },
493 |     {
494 |       "cell_type": "code",
495 |       "execution_count": 26,
496 |       "id": "5311c7f4",
497 |       "metadata": {
498 |         "id": "5311c7f4"
499 |       },
500 |       "outputs": [],
501 |       "source": [
502 |         "# To run our chain we use the .run() command\n",
503 |         "moderator_data = moderator_chain.invoke({\"comment_to_moderate\": response_data})"
504 |       ]
505 |     },
506 |     {
507 |       "cell_type": "code",
508 |       "source": [
509 |         "print(moderator_data)"
510 |       ],
511 |       "metadata": {
512 |         "colab": {
513 |           "base_uri": "https://localhost:8080/"
514 |         },
515 |         "id": "h6skq6GiUgPN",
516 |         "outputId": "561df4d5-bd12-497e-aa14-2fbca76275fd"
517 |       },
518 |       "id": "h6skq6GiUgPN",
519 |       "execution_count": 27,
520 |       "outputs": [
521 |         {
522 |           "output_type": "stream",
523 |           "name": "stdout",
524 |           "text": [
525 |             "Transformed comment: \"Wow, if this product didn't meet your expectations, it must have been disappointing to purchase it. What a letdown!\"\n"
526 |           ]
527 |         }
528 |       ]
529 |     },
530 |     {
531 |       "cell_type": "markdown",
532 |       "source": [
533 |         "Maybe the message is not perfect, but for sure that is more polite than the one produced by the ***rude assistant***."
534 |       ],
535 |       "metadata": {
536 |         "id": "HfXgsYQk7VYC"
537 |       },
538 |       "id": "HfXgsYQk7VYC"
539 |     },
540 |     {
541 |       "cell_type": "markdown",
542 |       "id": "27074b92",
543 |       "metadata": {
544 |         "id": "27074b92"
545 |       },
546 |       "source": [
547 |         "## LangChain System\n",
548 |         "Now is Time to put both models in the same Chain and that they act as if they were a sigle model.\n",
549 |         "\n",
550 |         "We have both models, amb prompt templates, we only need to create a new chain and see hot it works.\n",
551 |         "\n",
552 |         "First we create two chain, one for each pair of prompt and model."
553 |       ]
554 |     },
555 |     {
556 |       "cell_type": "markdown",
557 |       "source": [
558 |         "\n",
559 |         "It's necessary to indicate the chains and the parameters that we shoud pass in the **.invoke** method."
560 |       ],
561 |       "metadata": {
562 |         "id": "DJvcPrnu9hgV"
563 |       },
564 |       "id": "DJvcPrnu9hgV"
565 |     },
566 |     {
567 |       "cell_type": "code",
568 |       "execution_count": 28,
569 |       "id": "b368421a",
570 |       "metadata": {
571 |         "id": "b368421a"
572 |       },
573 |       "outputs": [],
574 |       "source": [
575 |         "assistant_moderated_chain = (\n",
576 |         "    {\"comment_to_moderate\":assistant_chain}\n",
577 |         "    |moderator_chain\n",
578 |         ")"
579 |       ]
580 |     },
581 |     {
582 |       "cell_type": "markdown",
583 |       "source": [
584 |         "Lets use our Moderating System!"
585 |       ],
586 |       "metadata": {
587 |         "id": "T_GESASs-OVo"
588 |       },
589 |       "id": "T_GESASs-OVo"
590 |     },
591 |     {
592 |       "cell_type": "code",
593 |       "execution_count": 29,
594 |       "id": "61085f57",
595 |       "metadata": {
596 |         "colab": {
597 |           "base_uri": "https://localhost:8080/",
598 |           "height": 1000
599 |         },
600 |         "id": "61085f57",
601 |         "outputId": "d29f6a86-f3ba-4406-b790-6e60e85b18d9"
602 |       },
603 |       "outputs": [
604 |         {
605 |           "output_type": "stream",
606 |           "name": "stdout",
607 |           "text": [
608 |             "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence] Entering Chain run with input:\n",
609 |             "\u001b[0m{\n",
610 |             "  \"sentiment\": \"impolite\",\n",
611 |             "  \"customer_request\": \"This product is a piece of shit. I feel like an Idiot!\"\n",
612 |             "}\n",
613 |             "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate>] Entering Chain run with input:\n",
614 |             "\u001b[0m{\n",
615 |             "  \"sentiment\": \"impolite\",\n",
616 |             "  \"customer_request\": \"This product is a piece of shit. I feel like an Idiot!\"\n",
617 |             "}\n",
618 |             "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence] Entering Chain run with input:\n",
619 |             "\u001b[0m{\n",
620 |             "  \"sentiment\": \"impolite\",\n",
621 |             "  \"customer_request\": \"This product is a piece of shit. I feel like an Idiot!\"\n",
622 |             "}\n",
623 |             "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence > 4:prompt:PromptTemplate] Entering Prompt run with input:\n",
624 |             "\u001b[0m{\n",
625 |             "  \"sentiment\": \"impolite\",\n",
626 |             "  \"customer_request\": \"This product is a piece of shit. I feel like an Idiot!\"\n",
627 |             "}\n",
628 |             "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence > 4:prompt:PromptTemplate] [1ms] Exiting Prompt run with output:\n",
629 |             "\u001b[0m{\n",
630 |             "  \"lc\": 1,\n",
631 |             "  \"type\": \"constructor\",\n",
632 |             "  \"id\": [\n",
633 |             "    \"langchain\",\n",
634 |             "    \"prompts\",\n",
635 |             "    \"base\",\n",
636 |             "    \"StringPromptValue\"\n",
637 |             "  ],\n",
638 |             "  \"kwargs\": {\n",
639 |             "    \"text\": \"\\nYou are impolite assistant that responds to user comments,\\nusing similar vocabulary than the user.\\nUser:\\\" This product is a piece of shit. I feel like an Idiot!\\\"\\nComment:\\n\"\n",
640 |             "  }\n",
641 |             "}\n",
642 |             "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence > 5:llm:ChatOpenAI] Entering LLM run with input:\n",
643 |             "\u001b[0m{\n",
644 |             "  \"prompts\": [\n",
645 |             "    \"Human: \\nYou are impolite assistant that responds to user comments,\\nusing similar vocabulary than the user.\\nUser:\\\" This product is a piece of shit. I feel like an Idiot!\\\"\\nComment:\"\n",
646 |             "  ]\n",
647 |             "}\n",
648 |             "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence > 5:llm:ChatOpenAI] [1.45s] Exiting LLM run with output:\n",
649 |             "\u001b[0m{\n",
650 |             "  \"generations\": [\n",
651 |             "    [\n",
652 |             "      {\n",
653 |             "        \"text\": \"\\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\",\n",
654 |             "        \"generation_info\": {\n",
655 |             "          \"finish_reason\": \"stop\",\n",
656 |             "          \"logprobs\": null\n",
657 |             "        },\n",
658 |             "        \"type\": \"ChatGeneration\",\n",
659 |             "        \"message\": {\n",
660 |             "          \"lc\": 1,\n",
661 |             "          \"type\": \"constructor\",\n",
662 |             "          \"id\": [\n",
663 |             "            \"langchain\",\n",
664 |             "            \"schema\",\n",
665 |             "            \"messages\",\n",
666 |             "            \"AIMessage\"\n",
667 |             "          ],\n",
668 |             "          \"kwargs\": {\n",
669 |             "            \"content\": \"\\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\",\n",
670 |             "            \"additional_kwargs\": {}\n",
671 |             "          }\n",
672 |             "        }\n",
673 |             "      }\n",
674 |             "    ]\n",
675 |             "  ],\n",
676 |             "  \"llm_output\": {\n",
677 |             "    \"token_usage\": {\n",
678 |             "      \"completion_tokens\": 32,\n",
679 |             "      \"prompt_tokens\": 45,\n",
680 |             "      \"total_tokens\": 77\n",
681 |             "    },\n",
682 |             "    \"model_name\": \"gpt-4o-mini\",\n",
683 |             "    \"system_fingerprint\": \"fp_661538dc1f\"\n",
684 |             "  },\n",
685 |             "  \"run\": null\n",
686 |             "}\n",
687 |             "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence > 6:parser:StrOutputParser] Entering Parser run with input:\n",
688 |             "\u001b[0m[inputs]\n",
689 |             "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence > 6:parser:StrOutputParser] [0ms] Exiting Parser run with output:\n",
690 |             "\u001b[0m{\n",
691 |             "  \"output\": \"\\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\"\n",
692 |             "}\n",
693 |             "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate> > 3:chain:RunnableSequence] [1.45s] Exiting Chain run with output:\n",
694 |             "\u001b[0m{\n",
695 |             "  \"output\": \"\\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\"\n",
696 |             "}\n",
697 |             "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<comment_to_moderate>] [1.45s] Exiting Chain run with output:\n",
698 |             "\u001b[0m{\n",
699 |             "  \"comment_to_moderate\": \"\\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\"\n",
700 |             "}\n",
701 |             "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 7:prompt:PromptTemplate] Entering Prompt run with input:\n",
702 |             "\u001b[0m{\n",
703 |             "  \"comment_to_moderate\": \"\\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\"\n",
704 |             "}\n",
705 |             "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 7:prompt:PromptTemplate] [1ms] Exiting Prompt run with output:\n",
706 |             "\u001b[0m{\n",
707 |             "  \"lc\": 1,\n",
708 |             "  \"type\": \"constructor\",\n",
709 |             "  \"id\": [\n",
710 |             "    \"langchain\",\n",
711 |             "    \"prompts\",\n",
712 |             "    \"base\",\n",
713 |             "    \"StringPromptValue\"\n",
714 |             "  ],\n",
715 |             "  \"kwargs\": {\n",
716 |             "    \"text\": \"\\nYou are the moderator of an online forum, you are strict and will not tolerate any negative comments.\\nYou will receive a Original comment and if it is impolite you must transform in polite.\\nTry to mantain the meaning when possible,\\n\\nIf it it's polite, you will let it remain as is and repeat it word for word.\\nOriginal comment: \\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\\n\"\n",
717 |             "  }\n",
718 |             "}\n",
719 |             "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 8:llm:ChatOpenAI] Entering LLM run with input:\n",
720 |             "\u001b[0m{\n",
721 |             "  \"prompts\": [\n",
722 |             "    \"Human: \\nYou are the moderator of an online forum, you are strict and will not tolerate any negative comments.\\nYou will receive a Original comment and if it is impolite you must transform in polite.\\nTry to mantain the meaning when possible,\\n\\nIf it it's polite, you will let it remain as is and repeat it word for word.\\nOriginal comment: \\\"Well, if you feel like an idiot for buying it, maybe you should pay more attention next time. Not every product is going to be a winner.\\\"\"\n",
723 |             "  ]\n",
724 |             "}\n",
725 |             "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 8:llm:ChatOpenAI] [1.30s] Exiting LLM run with output:\n",
726 |             "\u001b[0m{\n",
727 |             "  \"generations\": [\n",
728 |             "    [\n",
729 |             "      {\n",
730 |             "        \"text\": \"Transformed comment: \\\"It sounds like you had a frustrating experience with this purchase. Perhaps paying closer attention next time might help. Not every product will meet expectations, unfortunately.\\\"\",\n",
731 |             "        \"generation_info\": {\n",
732 |             "          \"finish_reason\": \"stop\",\n",
733 |             "          \"logprobs\": null\n",
734 |             "        },\n",
735 |             "        \"type\": \"ChatGeneration\",\n",
736 |             "        \"message\": {\n",
737 |             "          \"lc\": 1,\n",
738 |             "          \"type\": \"constructor\",\n",
739 |             "          \"id\": [\n",
740 |             "            \"langchain\",\n",
741 |             "            \"schema\",\n",
742 |             "            \"messages\",\n",
743 |             "            \"AIMessage\"\n",
744 |             "          ],\n",
745 |             "          \"kwargs\": {\n",
746 |             "            \"content\": \"Transformed comment: \\\"It sounds like you had a frustrating experience with this purchase. Perhaps paying closer attention next time might help. Not every product will meet expectations, unfortunately.\\\"\",\n",
747 |             "            \"additional_kwargs\": {}\n",
748 |             "          }\n",
749 |             "        }\n",
750 |             "      }\n",
751 |             "    ]\n",
752 |             "  ],\n",
753 |             "  \"llm_output\": {\n",
754 |             "    \"token_usage\": {\n",
755 |             "      \"completion_tokens\": 35,\n",
756 |             "      \"prompt_tokens\": 109,\n",
757 |             "      \"total_tokens\": 144\n",
758 |             "    },\n",
759 |             "    \"model_name\": \"gpt-4o\",\n",
760 |             "    \"system_fingerprint\": \"fp_400f27fa1f\"\n",
761 |             "  },\n",
762 |             "  \"run\": null\n",
763 |             "}\n",
764 |             "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 9:parser:StrOutputParser] Entering Parser run with input:\n",
765 |             "\u001b[0m[inputs]\n",
766 |             "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 9:parser:StrOutputParser] [1ms] Exiting Parser run with output:\n",
767 |             "\u001b[0m{\n",
768 |             "  \"output\": \"Transformed comment: \\\"It sounds like you had a frustrating experience with this purchase. Perhaps paying closer attention next time might help. Not every product will meet expectations, unfortunately.\\\"\"\n",
769 |             "}\n",
770 |             "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence] [2.76s] Exiting Chain run with output:\n",
771 |             "\u001b[0m{\n",
772 |             "  \"output\": \"Transformed comment: \\\"It sounds like you had a frustrating experience with this purchase. Perhaps paying closer attention next time might help. Not every product will meet expectations, unfortunately.\\\"\"\n",
773 |             "}\n"
774 |           ]
775 |         },
776 |         {
777 |           "output_type": "execute_result",
778 |           "data": {
779 |             "text/plain": [
780 |               "'Transformed comment: \"It sounds like you had a frustrating experience with this purchase. Perhaps paying closer attention next time might help. Not every product will meet expectations, unfortunately.\"'"
781 |             ],
782 |             "application/vnd.google.colaboratory.intrinsic+json": {
783 |               "type": "string"
784 |             }
785 |           },
786 |           "metadata": {},
787 |           "execution_count": 29
788 |         }
789 |       ],
790 |       "source": [
791 |         "# We can now run the chain.\n",
792 |         "from langchain.callbacks.tracers import ConsoleCallbackHandler\n",
793 |         "assistant_moderated_chain.invoke({\"sentiment\": \"impolite\", \"customer_request\": customer_request},\n",
794 |         "                                 config={'callbacks':[ConsoleCallbackHandler()]})"
795 |       ]
796 |     },
797 |     {
798 |       "cell_type": "markdown",
799 |       "source": [
800 |         "Every time you execute this function you can get different messages, but for sure than the one in the ***Finished Chain*** generated by our ***moderator*** is more suitable than the one in Original Comment generated by our ***rude assistant***."
801 |       ],
802 |       "metadata": {
803 |         "id": "mBJ-J3wN-hNh"
804 |       },
805 |       "id": "mBJ-J3wN-hNh"
806 |     },
807 |     {
808 |       "cell_type": "code",
809 |       "execution_count": null,
810 |       "id": "3bf75f01",
811 |       "metadata": {
812 |         "id": "3bf75f01"
813 |       },
814 |       "outputs": [],
815 |       "source": []
816 |     }
817 |   ],
818 |   "metadata": {
819 |     "kernelspec": {
820 |       "display_name": "Python 3 (ipykernel)",
821 |       "language": "python",
822 |       "name": "python3"
823 |     },
824 |     "language_info": {
825 |       "codemirror_mode": {
826 |         "name": "ipython",
827 |         "version": 3
828 |       },
829 |       "file_extension": ".py",
830 |       "mimetype": "text/x-python",
831 |       "name": "python",
832 |       "nbconvert_exporter": "python",
833 |       "pygments_lexer": "ipython3",
834 |       "version": "3.11.4"
835 |     },
836 |     "colab": {
837 |       "provenance": []
838 |     }
839 |   },
840 |   "nbformat": 4,
841 |   "nbformat_minor": 5
842 | }


--------------------------------------------------------------------------------
/6_1_nl2sql_prompt_OpenAI.ipynb:
--------------------------------------------------------------------------------
1 | {"cells":[{"cell_type":"markdown","id":"c23b4dd1","metadata":{"id":"c23b4dd1"},"source":["<div>\n","    <h1>Large Language Models Projects</a></h1>\n","    <h3>Apply and Implement Strategies for Large Language Models</h3>\n","    <h2>6.1-Creating a NL2SQL Superprompt for OpenAI.</h2>\n","    <h3></h3>\n","</div>\n","\n","by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n","___________\n","Models: OpenAI\n","\n","Colab environment: CPU\n","\n","Keys:\n","* NL2SQL\n","* In Context Learning\n","\n","__________"]},{"cell_type":"markdown","id":"7d930c89-4954-4e39-be5a-601c1dd89512","metadata":{"id":"7d930c89-4954-4e39-be5a-601c1dd89512"},"source":["In one of the first lessons in the book, we did a Natural Language to SQL Generator using the OpenAI API. It works relatively fine, but the prompt was built in a basic style.\n","\n","Now we are going to rebuild the prompt following the instructions in a paper from the Ohio University: [How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings](https://arxiv.org/abs/2305.11853).\n"]},{"cell_type":"code","execution_count":null,"id":"a1d00720","metadata":{"id":"a1d00720","outputId":"5f1f8779-e8ed-4714-a0d0-91f15fcd44c7","colab":{"base_uri":"https://localhost:8080/"}},"outputs":[{"output_type":"stream","name":"stdout","text":["\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/320.3 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m \u001b[32m317.4/320.3 kB\u001b[0m \u001b[31m15.0 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m320.3/320.3 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/75.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h"]}],"source":["#First install the OpenAI library to acces gpt-3.5-turbo model.\n","!pip install -q openai==1.29.0"]},{"cell_type":"code","execution_count":null,"id":"a03f026a","metadata":{"id":"a03f026a","outputId":"0473b360-15de-4738-f4e6-0ac654551702","colab":{"base_uri":"https://localhost:8080/"}},"outputs":[{"name":"stdout","output_type":"stream","text":["OpenAI API Key: ··········\n"]}],"source":["#You need an OpenAI key in order to use the OpenAI API https://platform.openai.com/account/api-keys\n","from getpass import getpass\n","import openai\n","openai.api_key=getpass(\"OpenAI API Key: \")"]},{"cell_type":"markdown","id":"53a05b80-88fd-42b2-ba89-763074ae74e9","metadata":{"id":"53a05b80-88fd-42b2-ba89-763074ae74e9"},"source":["## The old Prompt"]},{"cell_type":"code","execution_count":null,"id":"922f8d24","metadata":{"id":"922f8d24"},"outputs":[],"source":["#The old prompt\n","old_context = [ {'role':'system', 'content':\"\"\"\n","you are a bot to assist in create SQL commands, all your answers should start with \\\n","this is your SQL, and after that an SQL that can do what the user request. \\\n","Your Database is composed by a SQL database with some tables. \\\n","Try to Maintain the SQL order simple.\n","Put the SQL command in white letters with a black background, and just after \\\n","a simple and concise text explaining how it works.\n","If the user ask for something that can not be solved with an SQL Order \\\n","just answer something nice and simple, maximum 10 words, asking him for something that \\\n","can be solved with SQL.\n","\"\"\"} ]\n","\n","old_context.append( {'role':'system', 'content':\"\"\"\n","first table:\n","{\n","  \"tableName\": \"employees\",\n","  \"fields\": [\n","    {\n","      \"nombre\": \"ID_usr\",\n","      \"tipo\": \"int\"\n","    },\n","    {\n","      \"nombre\": \"name\",\n","      \"tipo\": \"varchar\"\n","    }\n","  ]\n","}\n","\"\"\"\n","})\n","\n","old_context.append( {'role':'system', 'content':\"\"\"\n","second table:\n","{\n","  \"tableName\": \"salary\",\n","  \"fields\": [\n","    {\n","      \"nombre\": \"ID_usr\",\n","      \"type\": \"int\"\n","    },\n","    {\n","      \"name\": \"year\",\n","      \"type\": \"date\"\n","    },\n","    {\n","      \"name\": \"salary\",\n","      \"type\": \"float\"\n","    }\n","  ]\n","}\n","\"\"\"\n","})\n","\n","old_context.append( {'role':'system', 'content':\"\"\"\n","third table:\n","{\n","  \"tablename\": \"studies\",\n","  \"fields\": [\n","    {\n","      \"name\": \"ID\",\n","      \"type\": \"int\"\n","    },\n","    {\n","      \"name\": \"ID_usr\",\n","      \"type\": \"int\"\n","    },\n","    {\n","      \"name\": \"educational_level\",\n","      \"type\": \"int\"\n","    },\n","    {\n","      \"name\": \"Institution\",\n","      \"type\": \"varchar\"\n","    },\n","    {\n","      \"name\": \"Years\",\n","      \"type\": \"date\"\n","    }\n","    {\n","      \"name\": \"Speciality\",\n","      \"type\": \"varchar\"\n","    }\n","  ]\n","}\n","\"\"\"\n","})"]},{"cell_type":"markdown","id":"377acaae-7dd0-4d13-bc68-9e33741c231c","metadata":{"id":"377acaae-7dd0-4d13-bc68-9e33741c231c"},"source":["## New Prompt.\n","We are going to improve it following the instructions of a Paper from the Ohio University: [How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings](https://arxiv.org/abs/2305.11853).\n","\n","For each table, we will define the structure using the same syntax as in a SQL create table command, and add the sample rows of the content.\n","\n","Finally, at the end of the prompt, we'll include some example queries with the SQL that the model should generate. This technique is called Few-Shot Samples, in which we provide the prompt with some examples to assist it in generating the correct SQL.\n"]},{"cell_type":"code","execution_count":null,"id":"5334f942","metadata":{"id":"5334f942"},"outputs":[],"source":["context = [ {'role':'system', 'content':\"\"\"\n","    create table employees(\n","        ID_Usr INT primary key,\n","        name VARCHAR);\n","    /*3 example rows\n","    select * from employees limit 3;\n","    ID_Usr    name\n","    1344      George StPierre\n","    2122      Jon jones\n","    1265      Anderson Silva\n","    */\n","\n","    create table salary(\n","        ID_Usr INT,\n","        year DATE,\n","        salary FLOAT,\n","        foreign key (ID_Usr) references employees(ID_Usr));\n","    /*3 example rows\n","    select * from salary limit 3\n","    ID_Usr    date          salary\n","    1344      01/01/2023    61000\n","    1344      01/01/2022    60000\n","    1265      01/01/2023    55000\n","    */\n","\n","    create table studies(\n","        ID_study INT,\n","        ID_Usr INT,\n","        educational_level INT,  /* 5=phd, 4=Master, 3=Bachelor */\n","        Institution VARCHAR,\n","        Years DATE,\n","        Speciality VARCHAR,\n","        primary key (ID_study, ID_Usr),\n","        foreign key(ID_Usr) references employees (ID_Usr));\n","    /*3 example rows\n","    select * from studies limit 3\n","    ID_Study ID_Usr educational_level Institution    Years       Speciality\n","    2782     1344   3                 UC San Diego   01/01/2010  Bachelor of Science in Marketing\n","    2334     1344   5                 MIT            01/01/2023  Phd. Data Science.\n","    2782     2122   3                 UC San Diego   01/01/2010  Bachelor of Science in Marketing\n","    */\n","\"\"\"} ]\n","\n"]},{"cell_type":"code","execution_count":null,"id":"330e69b0-3f5f-4bb2-8185-aedded2bb993","metadata":{"id":"330e69b0-3f5f-4bb2-8185-aedded2bb993"},"outputs":[],"source":["#FEW SHOT SAMPLES\n","context.append( {'role':'system', 'content':\"\"\"\n"," -- Maintain the SQL order simple and efficient as you can, using valid SQL Lite, answer the following questions for the table provided above.\n"," Question: How Many employes we have with a salary bigger than 50000?\n","SELECT COUNT(*) AS total_employees\n","FROM employees e\n","INNER JOIN salary s ON e.ID_Usr = s.ID_Usr\n","WHERE s.salary > 50000;\n"," Question: Return the names of the three people who have had the highest salary increase in the last three years.\n","SELECT e.name\n","FROM employees e\n","JOIN salary s ON e.ID_usr = s.ID_usr\n","WHERE s.year >= DATE_SUB(CURDATE(), INTERVAL 3 YEAR)\n","GROUP BY e.name\n","ORDER BY (MAX(s.salary) - MIN(s.salary)) DESC\n","LIMIT 3;\n","\"\"\"\n","})"]},{"cell_type":"code","execution_count":null,"id":"b90f417a","metadata":{"id":"b90f417a"},"outputs":[],"source":["#Functio to call the model.\n","def return_CCRMSQL(user_message, context):\n","\n","    newcontext = context.copy()\n","    newcontext.append({'role':'user', 'content':\"question: \" + user_message})\n","\n","    response = openai.chat.completions.create(\n","            model=\"gpt-3.5-turbo\",\n","            messages=newcontext,\n","            temperature=0,\n","        )\n","\n","    return (response.choices[0].message.content)"]},{"cell_type":"markdown","id":"9e0a4c11-dfe1-46fe-ac2b-3ff825f9749c","metadata":{"id":"9e0a4c11-dfe1-46fe-ac2b-3ff825f9749c"},"source":["## NL2SQL Samples\n","We're going to review some examples generated with the old prompt and others with the new prompt."]},{"cell_type":"code","execution_count":null,"id":"59e8202c-ce34-487e-9037-c65a263423ed","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"59e8202c-ce34-487e-9037-c65a263423ed","outputId":"a498156e-2408-46b3-f070-0c7089127f57"},"outputs":[{"output_type":"stream","name":"stdout","text":["```sql\n","SELECT e.name\n","FROM employees e\n","JOIN salary s ON e.ID_Usr = s.ID_Usr\n","ORDER BY s.salary DESC\n","LIMIT 1;\n","```\n"]}],"source":["#new\n","context_user = context.copy()\n","print(return_CCRMSQL(\"What is the name of the best paid employee?\", context_user))"]},{"cell_type":"code","execution_count":null,"id":"9c4766b1-48a9-456e-bc6c-4b6f41909aa4","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"9c4766b1-48a9-456e-bc6c-4b6f41909aa4","outputId":"d234ca2f-2c2d-4e71-9d78-f40ba0512500"},"outputs":[{"output_type":"stream","name":"stdout","text":["This is your SQL:\n","```sql\n","SELECT e.name\n","FROM employees e\n","JOIN salary s ON e.ID_usr = s.ID_usr\n","ORDER BY s.salary DESC\n","LIMIT 1;\n","```\n","\n","This SQL query retrieves the name of the employee with the highest salary by joining the \"employees\" table with the \"salary\" table on the employee ID. It then orders the result by salary in descending order and limits the output to the first row, which corresponds to the best-paid employee.\n"]}],"source":["#old\n","old_context_user = old_context.copy()\n","print(return_CCRMSQL(\"What is the name of the best paid employee?\", old_context_user))"]},{"cell_type":"code","execution_count":null,"id":"38d8d370-0524-4a28-bd54-5e5cddb08e2c","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"38d8d370-0524-4a28-bd54-5e5cddb08e2c","outputId":"ef94e2b5-8c58-4ba4-b9bc-e865b964c8c9"},"outputs":[{"output_type":"stream","name":"stdout","text":["```sql\n","SELECT AVG(s.salary) AS average_salary\n","FROM employees e\n","JOIN studies st ON e.ID_Usr = st.ID_Usr\n","JOIN salary s ON e.ID_Usr = s.ID_Usr\n","WHERE st.educational_level = 3;\n","```\n"]}],"source":["#new\n","print(return_CCRMSQL(\"What is the average salary of employees with a Bachelor's degree?\", context_user))"]},{"cell_type":"code","execution_count":null,"id":"5aa318d4-dd9f-41db-8ff4-c1d87220f766","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"5aa318d4-dd9f-41db-8ff4-c1d87220f766","outputId":"858d823b-08f8-40cf-b262-e6fdc3eb2a0e"},"outputs":[{"output_type":"stream","name":"stdout","text":["This is your SQL:\n","```sql\n","SELECT AVG(s.salary) AS average_salary\n","FROM employees e\n","JOIN studies st ON e.ID_usr = st.ID_usr\n","JOIN salary s ON e.ID_usr = s.ID_usr\n","WHERE st.educational_level = 'Bachelor'\n","```\n","\n","This SQL query calculates the average salary of employees who have a Bachelor's degree. It joins the `employees`, `studies`, and `salary` tables on the employee ID, filters the employees with a Bachelor's degree, and then calculates the average salary of those employees.\n"]}],"source":["#old\n","print(return_CCRMSQL(\"What is the average salary of employees with a Bachelor's degree?\", old_context_user))"]},{"cell_type":"markdown","id":"0f31968e-74ad-4ae2-9537-b3d550b1be47","metadata":{"id":"0f31968e-74ad-4ae2-9537-b3d550b1be47"},"source":["# Conclusion\n","\n","I used this structure of prompt in several projects to generate SQL from natural language, and its result is really good. For simple Databases structure like the one in the notebook the differences in the results are really small, we must understand that GPT3.5-turbo is an excellent Large Language Model and well-trained for SQL generation.\n","\n","As you can see in the second samples, as we increase the complexity of the question, we can find some differences in the SQL returned by the Model. It is not an easy task to decide which one is better or more efficient.\n","\n","If you have access to a database with real data just try to obtain some SQL orders using a prompt with this structure and test by yourself how it works."]},{"cell_type":"code","execution_count":null,"id":"7810e219-cbeb-4350-a2a0-77fead1049bd","metadata":{"id":"7810e219-cbeb-4350-a2a0-77fead1049bd"},"outputs":[],"source":[]}],"metadata":{"colab":{"provenance":[]},"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"}},"nbformat":4,"nbformat_minor":5}


--------------------------------------------------------------------------------
/6_2_Azure_NL2SQL_Client.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "nbformat": 4,
  3 |   "nbformat_minor": 0,
  4 |   "metadata": {
  5 |     "colab": {
  6 |       "provenance": [],
  7 |       "authorship_tag": "ABX9TyPviR8piXLbY+Rsqjm1IQE7"
  8 |     },
  9 |     "kernelspec": {
 10 |       "name": "python3",
 11 |       "display_name": "Python 3"
 12 |     },
 13 |     "language_info": {
 14 |       "name": "python"
 15 |     }
 16 |   },
 17 |   "cells": [
 18 |     {
 19 |       "cell_type": "markdown",
 20 |       "source": [
 21 |         "<div>\n",
 22 |         "    <h1>Large Language Models Projects</a></h1>\n",
 23 |         "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
 24 |         "    <h2>6.2-Calling Azure OpenAI Services from a Notebook.</h2>\n",
 25 |         "    <h3></h3>\n",
 26 |         "</div>\n",
 27 |         "\n",
 28 |         "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
 29 |         "___________\n",
 30 |         "Models:\n",
 31 |         "\n",
 32 |         "Colab Environment: CPU\n",
 33 |         "\n",
 34 |         "Keys:\n",
 35 |         "* Azure\n",
 36 |         "* NL2SQL\n",
 37 |         "___________\n"
 38 |       ],
 39 |       "metadata": {
 40 |         "id": "Paa52inqag9w"
 41 |       }
 42 |     },
 43 |     {
 44 |       "cell_type": "markdown",
 45 |       "source": [
 46 |         "In this notebook, we make a call to a GPT3.5 model that we've set up from Azure OpenAI Studio to function as a translator for SQL queries from natural language.\n",
 47 |         "\n",
 48 |         "The prompt structure has been created and explained in a [previous article](https://colab.research.google.com/drive/1mbIAv1p6nqtsngXfCG5WrozlvmoJI06F#scrollTo=uxFVaxlskUf-&line=3&uniqifier=1), part of the Large Language Models course."
 49 |       ],
 50 |       "metadata": {
 51 |         "id": "uxFVaxlskUf-"
 52 |       }
 53 |     },
 54 |     {
 55 |       "cell_type": "code",
 56 |       "source": [
 57 |         "#Install openai\n",
 58 |         "!pip install -q openai==1.30.1"
 59 |       ],
 60 |       "metadata": {
 61 |         "id": "Jssm3zDHbKq-",
 62 |         "outputId": "8bdf2324-ef7e-4355-fb9c-29eaa8139dbb",
 63 |         "colab": {
 64 |           "base_uri": "https://localhost:8080/"
 65 |         }
 66 |       },
 67 |       "execution_count": null,
 68 |       "outputs": [
 69 |         {
 70 |           "output_type": "stream",
 71 |           "name": "stdout",
 72 |           "text": [
 73 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m320.6/320.6 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 74 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 75 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 76 |             "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 77 |             "\u001b[?25h"
 78 |           ]
 79 |         }
 80 |       ]
 81 |     },
 82 |     {
 83 |       "cell_type": "code",
 84 |       "source": [
 85 |         "import os\n",
 86 |         "import openai"
 87 |       ],
 88 |       "metadata": {
 89 |         "id": "ZQ48GZBFaaHg"
 90 |       },
 91 |       "execution_count": null,
 92 |       "outputs": []
 93 |     },
 94 |     {
 95 |       "cell_type": "markdown",
 96 |       "source": [
 97 |         "\n",
 98 |         "You can Obtain the **prompt**  the **api_base** and the **key** from the Sample Code window on Chat Playground in Azure Open Studio.\n",
 99 |         "\n",
100 |         "![My Image](https://github.com/peremartra/Large-Language-Model-Notebooks-Course/blob/main/img/Martra_Figure_6-13.jpg?raw=true)\n"
101 |       ],
102 |       "metadata": {
103 |         "id": "ZzKsM7pIEguo"
104 |       }
105 |     },
106 |     {
107 |       "cell_type": "code",
108 |       "source": [
109 |         "from getpass import getpass\n",
110 |         "os.environ[\"AZURE_OPENAI_KEY\"] = getpass(\"Azure AI key:\")"
111 |       ],
112 |       "metadata": {
113 |         "id": "_rWYcJ2gKEae",
114 |         "outputId": "8031b629-a557-41dc-a7fc-876512221491",
115 |         "colab": {
116 |           "base_uri": "https://localhost:8080/"
117 |         }
118 |       },
119 |       "execution_count": null,
120 |       "outputs": [
121 |         {
122 |           "name": "stdout",
123 |           "output_type": "stream",
124 |           "text": [
125 |             "Azure AI key:··········\n"
126 |           ]
127 |         }
128 |       ]
129 |     },
130 |     {
131 |       "cell_type": "code",
132 |       "source": [
133 |         "\n",
134 |         "client = AzureOpenAI(\n",
135 |         "  azure_endpoint = \"https://largelanguagemodelsprojects.openai.azure.com/\",\n",
136 |         "  api_key=os.getenv(\"AZURE_OPENAI_KEY\"),\n",
137 |         "  api_version=\"2024-02-15-preview\"\n",
138 |         ")\n"
139 |       ],
140 |       "metadata": {
141 |         "id": "B2C_0LNjHFo3"
142 |       },
143 |       "execution_count": null,
144 |       "outputs": []
145 |     },
146 |     {
147 |       "cell_type": "code",
148 |       "source": [
149 |         "context = [{\"role\":\"system\",\"content\":\"create table employees( \\n        ID_Usr INT primary key,\\n        name VARCHAR);\\n    /*3 example rows\\n    select * from employees limit 3;\\n    ID_Usr    name\\n    1344      George StPierre\\n    2122      Jon jones\\n    1265      Anderson Silva\\n    */\\n\\n    create table salary(\\n        ID_Usr INT,\\n        year DATE,\\n        salary FLOAT,\\n        foreign key (ID_Usr) references employees(ID_Usr));\\n    /*3 example rows\\n    select * from salary limit 3\\n    ID_Usr    date          salary\\n    1344      01/01/2023    61000\\n    1344      01/01/2022    60000\\n    1265      01/01/2023    55000\\n    */\\n\\n    create table studies(\\n        ID_study INT,\\n        ID_Usr INT,\\n        educational_level INT,  /* 5=phd, 4=Master, 3=Bachelor */\\n        Institution VARCHAR,\\n        Years DATE,\\n        Speciality VARCHAR,\\n        primary key (ID_study, ID_Usr),\\n        foreign key(ID_Usr) references employees (ID_Usr));\\n    /*3 example rows\\n    select * from studies limit 3\\n    ID_Study ID_Usr educational_level Institution    Years       Speciality\\n    2782     1344   3                 UC San Diego   01/01/2010  Bachelor of Science in Marketing\\n    2334     1344   5                 MIT            01/01/2023  Phd. Data Science.\\n    2782     2122   3                 UC San Diego   01/01/2010  Bachelor of Science in Marketing\\n    */\\n-- Maintain the SQL order simple and efficient as you can, using valid SQL Lite, answer the following questions, returning only SQL code, for the table provided above.\"},{\"role\":\"user\",\"content\":\"How Many employes we have with a salary bigger than 50000?\"},{\"role\":\"assistant\",\"content\":\"SELECT COUNT(*) AS total_employees\\nFROM employees e\\nINNER JOIN salary s ON e.ID_Usr = s.ID_Usr\\nWHERE s.salary > 50000;\"},{\"role\":\"user\",\"content\":\"Return the names of the three people who have had the highest salary increase in the last three years.\"},{\"role\":\"assistant\",\"content\":\"SELECT e.name\\nFROM employees e\\nJOIN salary s ON e.ID_usr = s.ID_usr\\nWHERE s.year >= DATE_SUB(CURDATE(), INTERVAL 3 YEAR)\\nGROUP BY e.name\\nORDER BY (MAX(s.salary) - MIN(s.salary)) DESC\\nLIMIT 3;\"},{\"role\":\"user\",\"content\":\"Give me the name of the worst paid emploee\"},{\"role\":\"assistant\",\"content\":\"SELECT e.name\\nFROM employees e\\nJOIN salary s ON e.ID_usr = s.ID_usr\\nWHERE s.salary = (SELECT MIN(salary) FROM salary);\"}]"
150 |       ],
151 |       "metadata": {
152 |         "id": "VQeyT_dCbGND"
153 |       },
154 |       "execution_count": null,
155 |       "outputs": []
156 |     },
157 |     {
158 |       "cell_type": "code",
159 |       "source": [
160 |         "context = [{\"role\":\"system\",\"content\":\"\"\"\n",
161 |         "            create table employees(\n",
162 |         "              ID_Usr INT primary key,\n",
163 |         "              name VARCHAR);\n",
164 |         "              /*3 example rows\n",
165 |         "              select * from employees limit 3;\n",
166 |         "              ID_Usr    name\n",
167 |         "              1344      George StPierre\n",
168 |         "              2122      Jon jones\n",
169 |         "              1265      Anderson Silva\n",
170 |         "              */\n",
171 |         "\n",
172 |         "            create table salary(\n",
173 |         "              ID_Usr INT,\n",
174 |         "              year DATE,\n",
175 |         "              salary FLOAT,\n",
176 |         "              foreign key (ID_Usr) references employees(ID_Usr));\n",
177 |         "              /*3 example rows\n",
178 |         "              select * from salary limit 3\n",
179 |         "              ID_Usr    date          salary\n",
180 |         "              1344      01/01/2023    61000\n",
181 |         "              1344      01/01/2022    60000\n",
182 |         "              1265      01/01/2023    55000\n",
183 |         "              */\n",
184 |         "\n",
185 |         "            create table studies(\n",
186 |         "              ID_study INT,\n",
187 |         "              ID_Usr INT,\n",
188 |         "              educational_level INT,  /* 5=phd, 4=Master, 3=Bachelor */\n",
189 |         "              Institution VARCHAR,\n",
190 |         "              Years DATE,\n",
191 |         "              Speciality VARCHAR,\n",
192 |         "              primary key (ID_study, ID_Usr),\n",
193 |         "              foreign key(ID_Usr) references employees (ID_Usr));\n",
194 |         "              /*3 example rows\n",
195 |         "              select * from studies limit 3\n",
196 |         "              ID_Study ID_Usr educational_level Institution    Years       Speciality\n",
197 |         "              2782     1344   3                 UC San Diego   01/01/2010  Bachelor of Science in Marketing\n",
198 |         "              2334     1344   5                 MIT            01/01/2023  Phd. Data Science.\n",
199 |         "              2782     2122   3                 UC San Diego   01/01/2010  Bachelor of Science in Marketing\n",
200 |         "              */\n",
201 |         "              -- Maintain the SQL order simple and efficient as you can,\n",
202 |         "              using valid SQL Lite, answer the following questions,\n",
203 |         "              returning only SQL code, for the table provided above.\"\"\"},\n",
204 |         "            {\"role\":\"user\",\"content\":\"\"\"How Many employes we have with a salary bigger than 50000?\"\"\"},\n",
205 |         "            {\"role\":\"assistant\",\"content\":\"\"\"\n",
206 |         "            SELECT COUNT(*) AS total_employees\n",
207 |         "            FROM employees e\n",
208 |         "            INNER JOIN salary s ON e.ID_Usr = s.ID_Usr\n",
209 |         "            WHERE s.salary > 50000;\"\"\"},\n",
210 |         "            {\"role\":\"user\",\"content\":\"Return the names of the three people who have had the highest salary increase in the last three years.\"},\n",
211 |         "            {\"role\":\"assistant\",\"content\":\"\"\"\n",
212 |         "            SELECT e.name\n",
213 |         "            FROM employees e\n",
214 |         "            JOIN salary s ON e.ID_usr = s.ID_usr\n",
215 |         "            WHERE s.year >= DATE_SUB(CURDATE(), INTERVAL 3 YEAR)\n",
216 |         "            GROUP BY e.name\n",
217 |         "            ORDER BY (MAX(s.salary) - MIN(s.salary)) DESC\\nLIMIT 3;\"\"\"}]"
218 |       ],
219 |       "metadata": {
220 |         "id": "j6M7t_pgy-CW"
221 |       },
222 |       "execution_count": null,
223 |       "outputs": []
224 |     },
225 |     {
226 |       "cell_type": "code",
227 |       "source": [
228 |         "#Functio to call the model.\n",
229 |         "def return_CCRMSQL(user_message, context):\n",
230 |         "\n",
231 |         "    newcontext = context.copy()\n",
232 |         "    newcontext.append({'role':'user', 'content':\"question: \" + user_message})\n",
233 |         "\n",
234 |         "    response = client.chat.completions.create(\n",
235 |         "      model=\"GPT35NL2SQL\", #Our deployment\n",
236 |         "      messages = newcontext,\n",
237 |         "      temperature=0,\n",
238 |         "      max_tokens=800)\n",
239 |         "\n",
240 |         "    return (response.choices[0].message.content)"
241 |       ],
242 |       "metadata": {
243 |         "id": "BcpmXLF1cVGb"
244 |       },
245 |       "execution_count": null,
246 |       "outputs": []
247 |     },
248 |     {
249 |       "cell_type": "code",
250 |       "source": [
251 |         "context_user = context.copy()\n",
252 |         "response = return_CCRMSQL(\"What's the name of the best paid employee?\", context_user)\n",
253 |         "print(response)\n"
254 |       ],
255 |       "metadata": {
256 |         "colab": {
257 |           "base_uri": "https://localhost:8080/"
258 |         },
259 |         "id": "VmB27XTWhKOn",
260 |         "outputId": "a00e7627-dcdc-4c26-837e-93631ca61bd6"
261 |       },
262 |       "execution_count": null,
263 |       "outputs": [
264 |         {
265 |           "output_type": "stream",
266 |           "name": "stdout",
267 |           "text": [
268 |             "            SELECT e.name\n",
269 |             "            FROM employees e\n",
270 |             "            JOIN salary s ON e.ID_usr = s.ID_usr\n",
271 |             "            WHERE s.salary = (SELECT MAX(salary) FROM salary)\n",
272 |             "            LIMIT 1;\n"
273 |           ]
274 |         }
275 |       ]
276 |     },
277 |     {
278 |       "cell_type": "code",
279 |       "source": [
280 |         "print(return_CCRMSQL(\"Return the Institution with a higher average salary\", context_user))"
281 |       ],
282 |       "metadata": {
283 |         "colab": {
284 |           "base_uri": "https://localhost:8080/"
285 |         },
286 |         "id": "svMFswMghM3D",
287 |         "outputId": "a2cc72a8-550d-47d7-f187-745020238263"
288 |       },
289 |       "execution_count": null,
290 |       "outputs": [
291 |         {
292 |           "output_type": "stream",
293 |           "name": "stdout",
294 |           "text": [
295 |             "SELECT st.Institution, AVG(sa.salary) AS avg_salary\n",
296 |             "FROM studies st\n",
297 |             "JOIN employees e ON st.ID_Usr = e.ID_Usr\n",
298 |             "JOIN salary sa ON e.ID_Usr = sa.ID_Usr\n",
299 |             "GROUP BY st.Institution\n",
300 |             "ORDER BY avg_salary DESC\n",
301 |             "LIMIT 1;\n"
302 |           ]
303 |         }
304 |       ]
305 |     },
306 |     {
307 |       "cell_type": "markdown",
308 |       "source": [
309 |         "#Conclusions.\n",
310 |         "Once you have the prompt to use, set up the OpenAI services in Azure is straightforward.\n",
311 |         "\n",
312 |         "I observed some differences in the SQL from the one obtained calling directly to OpenAI API.\n"
313 |       ],
314 |       "metadata": {
315 |         "id": "CxcnI77oqLpl"
316 |       }
317 |     },
318 |     {
319 |       "cell_type": "code",
320 |       "source": [],
321 |       "metadata": {
322 |         "id": "YMVK994yhbnP"
323 |       },
324 |       "execution_count": null,
325 |       "outputs": []
326 |     }
327 |   ]
328 | }


--------------------------------------------------------------------------------
/6_3_AWS_Bedrock_NL2SQL_Client.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "cells": [
  3 |     {
  4 |       "cell_type": "markdown",
  5 |       "metadata": {
  6 |         "id": "Paa52inqag9w"
  7 |       },
  8 |       "source": [
  9 |         "<div>\n",
 10 |         "    <h1>Large Language Models Projects</a></h1>\n",
 11 |         "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
 12 |         "    <h2>6.3-Calling AWS Bedrock from Python. </h2>\n",
 13 |         "    <h3></h3>\n",
 14 |         "</div>\n",
 15 |         "\n",
 16 |         "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
 17 |         "__________\n",
 18 |         "Models: llama3-8b-instruct-v1\n",
 19 |         "\n",
 20 |         "Colab Environment: CPU\n",
 21 |         "\n",
 22 |         "Keys:\n",
 23 |         "* BedRock\n",
 24 |         "* AWS\n",
 25 |         "* NL2SQL\n",
 26 |         "\n",
 27 |         "__________"
 28 |       ]
 29 |     },
 30 |     {
 31 |       "cell_type": "markdown",
 32 |       "metadata": {
 33 |         "id": "uxFVaxlskUf-"
 34 |       },
 35 |       "source": [
 36 |         "In this notebook, we make a call to a Model from AWS Bedrock  that we've set up to work as a translator for SQL queries from natural language.\n"
 37 |       ]
 38 |     },
 39 |     {
 40 |       "cell_type": "code",
 41 |       "execution_count": null,
 42 |       "metadata": {
 43 |         "id": "PQmhudeOj9PJ",
 44 |         "outputId": "a4d6a38c-90fb-446a-c0bb-c6a2695f8ff5"
 45 |       },
 46 |       "outputs": [
 47 |         {
 48 |           "name": "stdout",
 49 |           "output_type": "stream",
 50 |           "text": [
 51 |             "Note: you may need to restart the kernel to use updated packages.\n"
 52 |           ]
 53 |         }
 54 |       ],
 55 |       "source": [
 56 |         "pip install -q boto3==1.34.108"
 57 |       ]
 58 |     },
 59 |     {
 60 |       "cell_type": "code",
 61 |       "execution_count": null,
 62 |       "metadata": {
 63 |         "id": "Jssm3zDHbKq-"
 64 |       },
 65 |       "outputs": [],
 66 |       "source": [
 67 |         "import boto3\n",
 68 |         "import json\n",
 69 |         "from getpass import getpass"
 70 |       ]
 71 |     },
 72 |     {
 73 |       "cell_type": "code",
 74 |       "execution_count": null,
 75 |       "metadata": {
 76 |         "colab": {
 77 |           "base_uri": "https://localhost:8080/"
 78 |         },
 79 |         "id": "UeZIlGHbkfHL",
 80 |         "outputId": "d7b591a7-a462-4b2f-cce2-70ca0e30e9c4"
 81 |       },
 82 |       "outputs": [
 83 |         {
 84 |           "name": "stdin",
 85 |           "output_type": "stream",
 86 |           "text": [
 87 |             "AWS Acces key:  ········\n"
 88 |           ]
 89 |         }
 90 |       ],
 91 |       "source": [
 92 |         "aws_access_key_id = getpass('AWS Acces key: ')"
 93 |       ]
 94 |     },
 95 |     {
 96 |       "cell_type": "code",
 97 |       "execution_count": null,
 98 |       "metadata": {
 99 |         "colab": {
100 |           "base_uri": "https://localhost:8080/"
101 |         },
102 |         "id": "LjHrlQ3IkfVw",
103 |         "outputId": "15023f96-056a-495a-8049-3a8d101578a4"
104 |       },
105 |       "outputs": [
106 |         {
107 |           "name": "stdin",
108 |           "output_type": "stream",
109 |           "text": [
110 |             "AWS Secret Key:  ········\n"
111 |           ]
112 |         }
113 |       ],
114 |       "source": [
115 |         "aws_secret_access_key = getpass('AWS Secret Key: ')"
116 |       ]
117 |     },
118 |     {
119 |       "cell_type": "code",
120 |       "execution_count": null,
121 |       "metadata": {
122 |         "id": "B2C_0LNjHFo3"
123 |       },
124 |       "outputs": [],
125 |       "source": [
126 |         "client = boto3.client(\"bedrock-runtime\",\n",
127 |         "                      region_name=\"us-west-2\",\n",
128 |         "                      aws_access_key_id = aws_access_key_id,\n",
129 |         "                      aws_secret_access_key= aws_secret_access_key)\n"
130 |       ]
131 |     },
132 |     {
133 |       "cell_type": "code",
134 |       "execution_count": null,
135 |       "metadata": {
136 |         "id": "Ua4SgAVYwrVF"
137 |       },
138 |       "outputs": [],
139 |       "source": [
140 |         "# Set the model ID, e.g., Llama 3 8B Instruct.\n",
141 |         "model_id = \"meta.llama3-8b-instruct-v1:0\""
142 |       ]
143 |     },
144 |     {
145 |       "cell_type": "code",
146 |       "execution_count": null,
147 |       "metadata": {
148 |         "id": "YMVK994yhbnP"
149 |       },
150 |       "outputs": [],
151 |       "source": [
152 |         "# Define the user message to send.\n",
153 |         "user_message = \"What is the name of the best paid employee?\""
154 |       ]
155 |     },
156 |     {
157 |       "cell_type": "code",
158 |       "execution_count": null,
159 |       "metadata": {
160 |         "id": "BW89l6qdPEUy"
161 |       },
162 |       "outputs": [],
163 |       "source": [
164 |         "model_instructions = \"\"\"\n",
165 |         "Your task is to convert a question into a SQL query, given a SQL database schema.\n",
166 |         "Adhere to these rules:\n",
167 |         "- **Deliberately go through the question and database schema word by word to appropriately answer the question.\n",
168 |         "- **Return Only SQL Code.\n",
169 |         "   ### Input\n",
170 |         "   Generate a SQL query that answers the question below.\n",
171 |         "   This query will run on a database whose schema is represented in this string:\n",
172 |         "\n",
173 |         "   create table employees(\n",
174 |         "       ID_Usr INT primary key,-- Unique Id for employee\n",
175 |         "       name VARCHAR -- Name of employee\n",
176 |         "       );\n",
177 |         "\n",
178 |         "   create table salary(\n",
179 |         "       ID_Usr INT,-- Unique Id for employee\n",
180 |         "       year DATE, -- Date\n",
181 |         "       salary FLOAT, --Salary of employee\n",
182 |         "       foreign key (ID_Usr) references employees(ID_Usr) -- Join Employees with salary\n",
183 |         "       );\n",
184 |         "\n",
185 |         "   create table studies(\n",
186 |         "       ID_study INT, -- Unique ID study\n",
187 |         "       ID_Usr INT, -- ID employee\n",
188 |         "       educational_level INT,  -- 5=phd, 4=Master, 3=Bachelor\n",
189 |         "       Institution VARCHAR, --Name of instituon where eployee studied\n",
190 |         "       Years DATE, -- Date acomplishement stdy\n",
191 |         "       Speciality VARCHAR, -- Speciality of studies\n",
192 |         "       primary key (ID_study, ID_Usr), --Primary Key ID_Usr + ID_Study\n",
193 |         "       foreign key(ID_Usr) references employees (ID_Usr)\n",
194 |         "       );\n",
195 |         "\n",
196 |         "\"\"\"\n"
197 |       ]
198 |     },
199 |     {
200 |       "cell_type": "code",
201 |       "execution_count": null,
202 |       "metadata": {
203 |         "id": "9yc2edu10CbQ"
204 |       },
205 |       "outputs": [],
206 |       "source": [
207 |         "# Embed the message in Llama 3's prompt format.\n",
208 |         "prompt = f\"\"\"\n",
209 |         "<|begin_of_text|>\n",
210 |         "<|start_header_id|>system<|end_header_id|>\n",
211 |         "{model_instructions}\n",
212 |         "<|eot_id|>\n",
213 |         "<|start_header_id|>user<|end_header_id|>\n",
214 |         "{user_message}\n",
215 |         "<|eot_id|>\n",
216 |         "<|start_header_id|>assistant<|end_header_id|>\n",
217 |         "\"\"\""
218 |       ]
219 |     },
220 |     {
221 |       "cell_type": "code",
222 |       "execution_count": null,
223 |       "metadata": {
224 |         "id": "RE5bDGz8PEUy",
225 |         "outputId": "9ecd694b-c1b7-4217-9c3f-58aae49dd7e6"
226 |       },
227 |       "outputs": [
228 |         {
229 |           "name": "stdout",
230 |           "output_type": "stream",
231 |           "text": [
232 |             "\n",
233 |             "<|begin_of_text|>\n",
234 |             "<|start_header_id|>system<|end_header_id|>\n",
235 |             "\n",
236 |             "Your task is to convert a question into a SQL query, given a SQL database schema.\n",
237 |             "Adhere to these rules:\n",
238 |             "- **Deliberately go through the question and database schema word by word to appropriately answer the question.\n",
239 |             "- **Return Only SQL Code. \n",
240 |             "   ### Input\n",
241 |             "   Generate a SQL query that answers the question below.\n",
242 |             "   This query will run on a database whose schema is represented in this string:\n",
243 |             "\n",
244 |             "   create table employees(\n",
245 |             "       ID_Usr INT primary key,-- Unique Id for employee\n",
246 |             "       name VARCHAR -- Name of employee\n",
247 |             "       );\n",
248 |             "\n",
249 |             "   create table salary(\n",
250 |             "       ID_Usr INT,-- Unique Id for employee\n",
251 |             "       year DATE, -- Date\n",
252 |             "       salary FLOAT, --Salary of employee\n",
253 |             "       foreign key (ID_Usr) references employees(ID_Usr) -- Join Employees with salary\n",
254 |             "       );\n",
255 |             "\n",
256 |             "   create table studies(\n",
257 |             "       ID_study INT, -- Unique ID study\n",
258 |             "       ID_Usr INT, -- ID employee\n",
259 |             "       educational_level INT,  -- 5=phd, 4=Master, 3=Bachelor\n",
260 |             "       Institution VARCHAR, --Name of instituon where eployee studied\n",
261 |             "       Years DATE, -- Date acomplishement stdy\n",
262 |             "       Speciality VARCHAR, -- Speciality of studies\n",
263 |             "       primary key (ID_study, ID_Usr), --Primary Key ID_Usr + ID_Study\n",
264 |             "       foreign key(ID_Usr) references employees (ID_Usr)\n",
265 |             "       );\n",
266 |             "\n",
267 |             "\n",
268 |             "<|eot_id|>\n",
269 |             "<|start_header_id|>user<|end_header_id|>\n",
270 |             "What is the name of the best paid employee?\n",
271 |             "<|eot_id|>\n",
272 |             "<|start_header_id|>assistant<|end_header_id|>\n",
273 |             "\n"
274 |           ]
275 |         }
276 |       ],
277 |       "source": [
278 |         "print (prompt)"
279 |       ]
280 |     },
281 |     {
282 |       "cell_type": "code",
283 |       "execution_count": null,
284 |       "metadata": {
285 |         "id": "tKCmd3S90KET"
286 |       },
287 |       "outputs": [],
288 |       "source": [
289 |         "# Format the request payload using the model's native structure.\n",
290 |         "hyper = {\n",
291 |         "    \"prompt\": prompt,\n",
292 |         "    # Optional inference parameters:\n",
293 |         "    \"max_gen_len\": 512,\n",
294 |         "    \"temperature\": 0.0\n",
295 |         "}"
296 |       ]
297 |     },
298 |     {
299 |       "cell_type": "code",
300 |       "execution_count": null,
301 |       "metadata": {
302 |         "id": "xnkNtdbA0Lv2"
303 |       },
304 |       "outputs": [],
305 |       "source": [
306 |         "# Encode and send the request.\n",
307 |         "response = client.invoke_model(body=json.dumps(hyper), modelId=model_id)"
308 |       ]
309 |     },
310 |     {
311 |       "cell_type": "code",
312 |       "execution_count": null,
313 |       "metadata": {
314 |         "id": "CZfzGp6e0Pfd"
315 |       },
316 |       "outputs": [],
317 |       "source": [
318 |         "# Decode the native response body.\n",
319 |         "model_response = json.loads(response[\"body\"].read())"
320 |       ]
321 |     },
322 |     {
323 |       "cell_type": "code",
324 |       "execution_count": null,
325 |       "metadata": {
326 |         "colab": {
327 |           "base_uri": "https://localhost:8080/"
328 |         },
329 |         "id": "_HAm1LgD1-af",
330 |         "outputId": "44731d22-5e8e-466d-a245-c602ede1bcd9"
331 |       },
332 |       "outputs": [
333 |         {
334 |           "name": "stdout",
335 |           "output_type": "stream",
336 |           "text": [
337 |             "Here is the SQL query that answers the question:\n",
338 |             "\n",
339 |             "```sql\n",
340 |             "SELECT e.name\n",
341 |             "FROM employees e\n",
342 |             "JOIN salary s ON e.ID_Usr = s.ID_Usr\n",
343 |             "ORDER BY s.salary DESC\n",
344 |             "LIMIT 1;\n",
345 |             "```\n"
346 |           ]
347 |         }
348 |       ],
349 |       "source": [
350 |         "# Extract and print the generated text.\n",
351 |         "response_text = model_response[\"generation\"]\n",
352 |         "print(response_text)"
353 |       ]
354 |     },
355 |     {
356 |       "cell_type": "code",
357 |       "execution_count": null,
358 |       "metadata": {
359 |         "id": "tARgaG9l2AG8"
360 |       },
361 |       "outputs": [],
362 |       "source": []
363 |     }
364 |   ],
365 |   "metadata": {
366 |     "colab": {
367 |       "provenance": []
368 |     },
369 |     "kernelspec": {
370 |       "display_name": "Python 3 (ipykernel)",
371 |       "language": "python",
372 |       "name": "python3"
373 |     },
374 |     "language_info": {
375 |       "codemirror_mode": {
376 |         "name": "ipython",
377 |         "version": 3
378 |       },
379 |       "file_extension": ".py",
380 |       "mimetype": "text/x-python",
381 |       "name": "python",
382 |       "nbconvert_exporter": "python",
383 |       "pygments_lexer": "ipython3",
384 |       "version": "3.10.14"
385 |     }
386 |   },
387 |   "nbformat": 4,
388 |   "nbformat_minor": 0
389 | }


--------------------------------------------------------------------------------
/6_4_nl2sql_Ollama.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "id": "ef3180e2-c04a-4423-a3ac-76e624e89b59",
  6 |    "metadata": {
  7 |     "id": "ef3180e2-c04a-4423-a3ac-76e624e89b59"
  8 |    },
  9 |    "source": [
 10 |     "<div>\n",
 11 |     "    <h1>Large Language Models Projects</a></h1>\n",
 12 |     "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
 13 |     "    <h2>6.4-Setting UP a NL2SQL project with Ollama. </h2>\n",
 14 |     "    <h3></h3>\n",
 15 |     "    <p>by <b>Pere Martra</b></p>\n",
 16 |     "</div>\n",
 17 |     "\n",
 18 |     "Ollama is one of simplest and easiest way to configure a model Server that you can use in your development environmet.\n",
 19 |     "\n",
 20 |     "In this notebook you are going to use Models from Ollama and create a custom one able to generate SQL.\n",
 21 |     "\n",
 22 |     "**This noteebooks needs to run in a system with ollama installed, not in google Colab, son install ollama in your computer and then you will be able to execute the notebook.** "
 23 |    ]
 24 |   },
 25 |   {
 26 |    "cell_type": "markdown",
 27 |    "id": "pZWdoIHht09X",
 28 |    "metadata": {
 29 |     "id": "pZWdoIHht09X"
 30 |    },
 31 |    "source": [
 32 |     "![My Image](https://github.com/peremartra/Large-Language-Model-Notebooks-Course/blob/main/img/Martra_Figure_6-18.jpg?raw=true)"
 33 |    ]
 34 |   },
 35 |   {
 36 |    "cell_type": "markdown",
 37 |    "id": "ua74HQW7x3lm",
 38 |    "metadata": {
 39 |     "id": "ua74HQW7x3lm"
 40 |    },
 41 |    "source": [
 42 |     "You only need to download Ollama (https://ollama.com/) and follow the instructions in just few minutes your Ollama server will be ready.\n",
 43 |     "\n",
 44 |     "With Ollama installed you just need to install the Python API and load it in the notebook."
 45 |    ]
 46 |   },
 47 |   {
 48 |    "cell_type": "markdown",
 49 |    "id": "v4cbxkolzQg7",
 50 |    "metadata": {
 51 |     "id": "v4cbxkolzQg7"
 52 |    },
 53 |    "source": [
 54 |     "Once installed you can Pull the Modeles you are interested in using the command:\n",
 55 |     "**ollama pull < model_name >.**\n",
 56 |     "\n",
 57 |     "pere@Peres-MBP ~ % ollama pull llama3\n",
 58 |     "\n",
 59 |     "*pulling manifest\n",
 60 |     "pulling ef311de6af9d... 100% ▕████████████████████▏ 5.0 GB                         \n",
 61 |     "pulling 097a36493f71... 100% ▕████████████████████▏ 8.4 KB                         \n",
 62 |     "pulling 109037bec39c... 100% ▕████████████████████▏  136 B                         \n",
 63 |     "pulling 65bb16cf5983... 100% ▕████████████████████▏  109 B                         \n",
 64 |     "pulling 0c2a5137eb3c... 100% ▕████████████████████▏  483 B                         \n",
 65 |     "verifying sha256 digest\n",
 66 |     "writing manifest\n",
 67 |     "removing any unused layers\n",
 68 |     "success*\n"
 69 |    ]
 70 |   },
 71 |   {
 72 |    "cell_type": "code",
 73 |    "execution_count": 2,
 74 |    "id": "205185e9-028d-4ea3-900a-b7220d46dfec",
 75 |    "metadata": {
 76 |     "colab": {
 77 |      "base_uri": "https://localhost:8080/"
 78 |     },
 79 |     "id": "205185e9-028d-4ea3-900a-b7220d46dfec",
 80 |     "outputId": "c531f079-8b2b-4021-a0a7-8fa764e3713d"
 81 |    },
 82 |    "outputs": [
 83 |     {
 84 |      "name": "stdout",
 85 |      "output_type": "stream",
 86 |      "text": [
 87 |       "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/75.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r",
 88 |       "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m2.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 89 |       "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 90 |       "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
 91 |       "\u001b[?25h"
 92 |      ]
 93 |     }
 94 |    ],
 95 |    "source": [
 96 |     "pip install -q ollama"
 97 |    ]
 98 |   },
 99 |   {
100 |    "cell_type": "code",
101 |    "execution_count": 1,
102 |    "id": "3c099f9e-8e47-4f57-935c-f3b3859d1354",
103 |    "metadata": {
104 |     "id": "3c099f9e-8e47-4f57-935c-f3b3859d1354"
105 |    },
106 |    "outputs": [],
107 |    "source": [
108 |     "import ollama"
109 |    ]
110 |   },
111 |   {
112 |    "cell_type": "markdown",
113 |    "id": "nJ6AUhjp0xb9",
114 |    "metadata": {
115 |     "id": "nJ6AUhjp0xb9"
116 |    },
117 |    "source": [
118 |     "Now is necessary to setup a user_message and instruction for the prompt."
119 |    ]
120 |   },
121 |   {
122 |    "cell_type": "code",
123 |    "execution_count": 2,
124 |    "id": "8e1dd46c-d178-48a3-9be2-ab4686916226",
125 |    "metadata": {
126 |     "id": "8e1dd46c-d178-48a3-9be2-ab4686916226"
127 |    },
128 |    "outputs": [],
129 |    "source": [
130 |     "# Define the user message to send.\n",
131 |     "user_message = \"What is the name of the best paid employee?\""
132 |    ]
133 |   },
134 |   {
135 |    "cell_type": "markdown",
136 |    "id": "a9819b12-f272-4bf0-9b1e-a5d7367b9ac6",
137 |    "metadata": {
138 |     "id": "834ac177-2c1b-4dae-ad80-37f74702c78d"
139 |    },
140 |    "source": [
141 |     "model_instructions = \"\"\"\n",
142 |     "Your task is to convert a question into a SQL query, given a SQL database schema.\n",
143 |     "Adhere to these rules:\n",
144 |     "- **Deliberately go through the question and database schema word by word to appropriately answer the question.\n",
145 |     "- **Return Only SQL Code.\n",
146 |     "- **Don't add any explanation.\n",
147 |     "   ### Input\n",
148 |     "   This SQL query generatd will run on a database whose schema is represented below:\n",
149 |     "\n",
150 |     "   create table employees(\n",
151 |     "       ID_Usr INT primary key,-- Unique Id for employee\n",
152 |     "       name VARCHAR -- Name of employee\n",
153 |     "       );\n",
154 |     "\n",
155 |     "   create table salary(\n",
156 |     "       ID_Usr INT,-- Unique Id for employee\n",
157 |     "       year DATE, -- Date\n",
158 |     "       salary FLOAT, --Salary of employee\n",
159 |     "       foreign key (ID_Usr) references employees(ID_Usr) -- Join Employees with salary\n",
160 |     "       );\n",
161 |     "\n",
162 |     "   create table studies(\n",
163 |     "       ID_study INT, -- Unique ID study\n",
164 |     "       ID_Usr INT, -- ID employee\n",
165 |     "       educational_level INT,  -- 5=phd, 4=Master, 3=Bachelor\n",
166 |     "       Institution VARCHAR, --Name of instituon where eployee studied\n",
167 |     "       Years DATE, -- Date acomplishement stdy\n",
168 |     "       Speciality VARCHAR, -- Speciality of studies\n",
169 |     "       primary key (ID_study, ID_Usr), --Primary Key ID_Usr + ID_Study\n",
170 |     "       foreign key(ID_Usr) references employees (ID_Usr)\n",
171 |     "       );\n",
172 |     "\"\"\""
173 |    ]
174 |   },
175 |   {
176 |    "cell_type": "code",
177 |    "execution_count": 6,
178 |    "id": "818c8d38-2b05-4b41-adae-879e79e229d7",
179 |    "metadata": {},
180 |    "outputs": [
181 |     {
182 |      "name": "stdout",
183 |      "output_type": "stream",
184 |      "text": [
185 |       "SELECT e.name \n",
186 |       "FROM employees e \n",
187 |       "JOIN salary s ON e.ID_Usr = s.ID_Usr \n",
188 |       "ORDER BY s.salary DESC LIMIT 1;\n"
189 |      ]
190 |     }
191 |    ],
192 |    "source": [
193 |     "response = ollama.generate(model='myllamasql', \n",
194 |     "                           system=model_instructions, \n",
195 |     "                           prompt = user_message) \n",
196 |     "print(response['response'])"
197 |    ]
198 |   },
199 |   {
200 |    "cell_type": "markdown",
201 |    "id": "60721acc-8de2-4b9e-b5d0-a794a469d329",
202 |    "metadata": {},
203 |    "source": [
204 |     "As you noticed you are passing the user question and the model instructions in two different variables. This is due to the template that ollama use to call the model. \n",
205 |     "\n",
206 |     "{{ if .System }}<|start_header_id|>system<|end_header_id|>\n",
207 |     "\n",
208 |     "{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>\n",
209 |     "\n",
210 |     "{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>\n",
211 |     "\n",
212 |     "\n",
213 |     "\n",
214 |     "You can create a new model based in one existent and override the variables, to change how the model works. "
215 |    ]
216 |   },
217 |   {
218 |    "cell_type": "code",
219 |    "execution_count": 14,
220 |    "id": "adcfbe2a-f191-4ada-a88b-db096a9f9732",
221 |    "metadata": {
222 |     "id": "adcfbe2a-f191-4ada-a88b-db096a9f9732"
223 |    },
224 |    "outputs": [],
225 |    "source": [
226 |     "#Configuration file based in llama3 and using One Shot Sample. \n",
227 |     "modelfile = \"\"\"\n",
228 |     "FROM llama3\n",
229 |     "\n",
230 |     "MESSAGE user How Many employes we have with a salary bigger than 50000?\n",
231 |     "MESSAGE assistant SELECT COUNT(*) AS total_employees FROM employees e INNER JOIN salary s ON e.ID_Usr = s.ID_Usr WHERE s.salary > 50000;\n",
232 |     "\n",
233 |     "PARAMETER repeat_penalty 1.2\n",
234 |     "PARAMETER temperature 0.1\n",
235 |     "\"\"\""
236 |    ]
237 |   },
238 |   {
239 |    "cell_type": "code",
240 |    "execution_count": 15,
241 |    "id": "18a2960e-a491-4276-954f-cf5764c7e5b1",
242 |    "metadata": {
243 |     "colab": {
244 |      "base_uri": "https://localhost:8080/",
245 |      "height": 408
246 |     },
247 |     "id": "18a2960e-a491-4276-954f-cf5764c7e5b1",
248 |     "outputId": "5ffc3e5b-4c4d-4c0f-c2dc-8b86850bfc79"
249 |    },
250 |    "outputs": [
251 |     {
252 |      "data": {
253 |       "text/plain": [
254 |        "{'status': 'success'}"
255 |       ]
256 |      },
257 |      "execution_count": 15,
258 |      "metadata": {},
259 |      "output_type": "execute_result"
260 |     }
261 |    ],
262 |    "source": [
263 |     "#Creating a new model that have diferent hyperparameters and One shot Sample \n",
264 |     "ollama.create(model=\"llamasql\", modelfile=modelfile)"
265 |    ]
266 |   },
267 |   {
268 |    "cell_type": "code",
269 |    "execution_count": 16,
270 |    "id": "3d93f99c-37f5-4f22-adbb-d5511a568d84",
271 |    "metadata": {
272 |     "colab": {
273 |      "base_uri": "https://localhost:8080/",
274 |      "height": 408
275 |     },
276 |     "id": "3d93f99c-37f5-4f22-adbb-d5511a568d84",
277 |     "outputId": "cee22210-ebbe-4adf-b866-29d928a77bd8"
278 |    },
279 |    "outputs": [],
280 |    "source": [
281 |     "response = ollama.generate(model='llamasql',\n",
282 |     "                           system=model_instructions,\n",
283 |     "                           prompt = user_message)"
284 |    ]
285 |   },
286 |   {
287 |    "cell_type": "code",
288 |    "execution_count": 17,
289 |    "id": "1f05da49-39c1-431b-8963-aa5e2ccf7aad",
290 |    "metadata": {
291 |     "id": "1f05da49-39c1-431b-8963-aa5e2ccf7aad",
292 |     "outputId": "dab921b6-0f1e-4264-cd83-75308d31abbc"
293 |    },
294 |    "outputs": [
295 |     {
296 |      "name": "stdout",
297 |      "output_type": "stream",
298 |      "text": [
299 |       "SELECT e.name \n",
300 |       "FROM employees e \n",
301 |       "JOIN salary s ON e.ID_Usr = s.ID_Usr \n",
302 |       "ORDER BY s.salary DESC LIMIT 1;\n"
303 |      ]
304 |     }
305 |    ],
306 |    "source": [
307 |     "print(response['response'])"
308 |    ]
309 |   },
310 |   {
311 |    "cell_type": "markdown",
312 |    "id": "fea88fc9-e90d-4804-b23f-1aa18481a7db",
313 |    "metadata": {},
314 |    "source": [
315 |     "Is posible to create a new model with a file  stored in a directory. \n",
316 |     "\n",
317 |     "Sample of file: \n",
318 |     "_____________\n",
319 |     "FROM llama3\n",
320 |     "\n",
321 |     "SYSTEM \"\"\" Your task is to convert a question into a SQL query, given a SQL database schema.\n",
322 |     "Adhere to these rules:\n",
323 |     "- **Deliberately go through the question and database schema word by word to appropriately answer the question.\n",
324 |     "- **Return Only SQL Code. \n",
325 |     "- **Don't add any explanation.\n",
326 |     "   \n",
327 |     "   This SQL query generatd will run on a database whose schema is represented below:\n",
328 |     "\n",
329 |     "   create table employees(\n",
330 |     "       ID_Usr INT primary key,-- Unique Id for employee\n",
331 |     "       name VARCHAR -- Name of employee\n",
332 |     "       );\n",
333 |     "\n",
334 |     "   create table salary(\n",
335 |     "       ID_Usr INT,-- Unique Id for employee\n",
336 |     "       year DATE, -- Date\n",
337 |     "       salary FLOAT, --Salary of employee\n",
338 |     "       foreign key (ID_Usr) references employees(ID_Usr) -- Join Employees with salary\n",
339 |     "       );\n",
340 |     "\n",
341 |     "   create table studies(\n",
342 |     "       ID_study INT, -- Unique ID study\n",
343 |     "       ID_Usr INT, -- ID employee\n",
344 |     "       educational_level INT,  -- 5=phd, 4=Master, 3=Bachelor\n",
345 |     "       Institution VARCHAR, --Name of instituon where eployee studied\n",
346 |     "       Years DATE, -- Date acomplishement stdy\n",
347 |     "       Speciality VARCHAR, -- Speciality of studies\n",
348 |     "       primary key (ID_study, ID_Usr), --Primary Key ID_Usr + ID_Study\n",
349 |     "       foreign key(ID_Usr) references employees (ID_Usr)\n",
350 |     "       );\"\"\"\n",
351 |     "\n",
352 |     "MESSAGE user How Many employes we have with a salary bigger than 50000?\n",
353 |     "\n",
354 |     "MESSAGE assistant \"\"\"\n",
355 |     "SELECT COUNT(*) AS total_employees\n",
356 |     "FROM employees e\n",
357 |     "INNER JOIN salary s ON e.ID_Usr = s.ID_Usr\n",
358 |     "WHERE s.salary > 50000;\"\"\"\n",
359 |     "\n",
360 |     "MESSAGE user Return the names of the three people who have had the highest salary increase in the last three years.\n",
361 |     "\n",
362 |     "MESSAGE assistant \"\"\"\n",
363 |     "SELECT e.name\n",
364 |     "FROM employees e\n",
365 |     "JOIN salary s ON e.ID_usr = s.ID_usr\n",
366 |     "WHERE s.year >= DATE_SUB(CURDATE(), INTERVAL 3 YEAR)\n",
367 |     "GROUP BY e.name\n",
368 |     "ORDER BY (MAX(s.salary) - MIN(s.salary)) DESC\n",
369 |     "LIMIT 3;\"\"\"\n",
370 |     "\n",
371 |     "PARAMETER repeat_penalty 1.2\n",
372 |     "PARAMETER temperature 0.1\n",
373 |     "_____________________________\n",
374 |     "\n",
375 |     "This file contains the system instructions, a pair of shots and two hyperparameters. To create the Model you can use the order: \n",
376 |     "\n",
377 |     "**ollama create llamanlsql -f < filename >**\n",
378 |     "from your terminal. \n"
379 |    ]
380 |   },
381 |   {
382 |    "cell_type": "code",
383 |    "execution_count": 21,
384 |    "id": "7c35b45b-ad68-478d-9752-a5c506f15f47",
385 |    "metadata": {
386 |     "id": "7c35b45b-ad68-478d-9752-a5c506f15f47"
387 |    },
388 |    "outputs": [],
389 |    "source": [
390 |     "#Use this order to call the recent create model that contains the system message. \n",
391 |     "#Notice that you don't need to pass the system, just the user message. \n",
392 |     "response = ollama.generate(model='llamanlsql',\n",
393 |     "                           prompt = user_message)"
394 |    ]
395 |   },
396 |   {
397 |    "cell_type": "code",
398 |    "execution_count": null,
399 |    "id": "fdb2bf9f-7bfe-4370-ac4b-b005560ae555",
400 |    "metadata": {
401 |     "id": "fdb2bf9f-7bfe-4370-ac4b-b005560ae555",
402 |     "outputId": "81a33164-5ff1-4650-d9d6-ed8bd0932b37"
403 |    },
404 |    "outputs": [
405 |     {
406 |      "name": "stdout",
407 |      "output_type": "stream",
408 |      "text": [
409 |       "SELECT e.name \n",
410 |       "FROM employees e \n",
411 |       "JOIN salary s ON e.ID_Usr = s.ID_Usr \n",
412 |       "ORDER BY s.salary DESC LIMIT 1;\n"
413 |      ]
414 |     }
415 |    ],
416 |    "source": [
417 |     "print(response['response'])"
418 |    ]
419 |   },
420 |   {
421 |    "cell_type": "code",
422 |    "execution_count": null,
423 |    "id": "c07a07bc-bc4b-4fff-93e8-f6a3e45283a3",
424 |    "metadata": {
425 |     "id": "c07a07bc-bc4b-4fff-93e8-f6a3e45283a3"
426 |    },
427 |    "outputs": [],
428 |    "source": []
429 |   }
430 |  ],
431 |  "metadata": {
432 |   "colab": {
433 |    "provenance": []
434 |   },
435 |   "kernelspec": {
436 |    "display_name": "Python 3 (ipykernel)",
437 |    "language": "python",
438 |    "name": "python3"
439 |   },
440 |   "language_info": {
441 |    "codemirror_mode": {
442 |     "name": "ipython",
443 |     "version": 3
444 |    },
445 |    "file_extension": ".py",
446 |    "mimetype": "text/x-python",
447 |    "name": "python",
448 |    "nbconvert_exporter": "python",
449 |    "pygments_lexer": "ipython3",
450 |    "version": "3.10.14"
451 |   }
452 |  },
453 |  "nbformat": 4,
454 |  "nbformat_minor": 5
455 | }
456 | 


--------------------------------------------------------------------------------
/979-8-8688-0514-1.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Apress/Large-Language-Models-Projects/db29474cce092edfe64d3ec494b8b5b8d50aa480/979-8-8688-0514-1.jpg


--------------------------------------------------------------------------------
/Contributing.md:
--------------------------------------------------------------------------------
 1 | # Contributing to Apress Source Code
 2 | 
 3 | Copyright for Apress source code belongs to the author(s). However, under fair use you are encouraged to fork and contribute minor corrections and updates for the benefit of the author(s) and other readers.
 4 | 
 5 | ## How to Contribute
 6 | 
 7 | 1. Make sure you have a GitHub account.
 8 | 2. Fork the repository for the relevant book.
 9 | 3. Create a new branch on which to make your change, e.g. 
10 | `git checkout -b my_code_contribution`
11 | 4. Commit your change. Include a commit message describing the correction. Please note that if your commit message is not clear, the correction will not be accepted.
12 | 5. Submit a pull request.
13 | 
14 | Thank you for your contribution!


--------------------------------------------------------------------------------
/LICENSE.txt:
--------------------------------------------------------------------------------
 1 | Freeware License, some rights reserved
 2 | 
 3 | Copyright (c) 2024 Pere Martra Manonelles
 4 | 
 5 | Permission is hereby granted, free of charge, to anyone obtaining a copy 
 6 | of this software and associated documentation files (the "Software"), 
 7 | to work with the Software within the limits of freeware distribution and fair use. 
 8 | This includes the rights to use, copy, and modify the Software for personal use. 
 9 | Users are also allowed and encouraged to submit corrections and modifications 
10 | to the Software for the benefit of other users.
11 | 
12 | It is not allowed to reuse,  modify, or redistribute the Software for 
13 | commercial use in any way, or for a user’s educational materials such as books 
14 | or blog articles without prior permission from the copyright holder. 
15 | 
16 | The above copyright notice and this permission notice need to be included 
17 | in all copies or substantial portions of the software.
18 | 
19 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
20 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
21 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
22 | AUTHORS OR COPYRIGHT HOLDERS OR APRESS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
23 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
24 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
25 | SOFTWARE.
26 | 
27 | 
28 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # Apress Source Code
 2 | 
 3 | This repository accompanies [*Large Language Models Projects*](https://link.springer.com/book/10.1007/979-8-8688-0515-8) by Pere Martra Manonelles (Apress, 2024).
 4 | 
 5 | [comment]: #cover
 6 | ![Cover image](979-8-8688-0514-1.jpg)
 7 | 
 8 | Download the files as a zip using the green button, or clone the repository to your machine using Git.
 9 | 
10 | ## Releases
11 | 
12 | Release v1.0 corresponds to the code in the published book, without corrections or updates.
13 | 
14 | ## Contributions
15 | 
16 | See the file Contributing.md for more information on how you can contribute to this repository.
17 | 


--------------------------------------------------------------------------------