└── 2024
├── .gitignore
├── genai-philippines
├── arize.log
├── genai-philippines-with-outputs.ipynb
├── genai-philippines.ipynb
└── kg.html
├── georgian-genai-bootcamp
├── arize.log
├── georgian-genai-bootcamp-Copy1.ipynb
├── georgian-genai-bootcamp.ipynb
└── kg.html
├── mistplay
├── mistplay-talk.ipynb
└── network_query_engine.ipynb
├── mlops
└── mlops-rag-bootcamp.ipynb
├── otpp
├── mds
│ └── parsed.md
└── otpp.ipynb
├── rbc
├── TODO.txt
└── rbc-llm-workshop.ipynb
├── vector-oss-tools
├── arize.log
├── kg.html
└── vector-oss-tools.ipynb
├── vector-pe-lab
├── Introducing_LLamaParse_Parsing_Instructions.ipynb
├── manga
│ └── manga.pdf
├── mds
│ ├── instruction-parsed.md
│ ├── latex-instruction-parsed.md
│ └── parsed.md
├── network_retriever.ipynb
└── vector-pe-lab.ipynb
└── vector-rag-bootcamp
└── rag-bootcamp.ipynb
/2024/.gitignore:
--------------------------------------------------------------------------------
1 | data/
2 | networks_data/
3 | vector_data/
4 | .ipynb_checkpoints/
5 | .env
6 | client-env-files/
7 | lib/
8 |
--------------------------------------------------------------------------------
/2024/genai-philippines/kg.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
16 |
22 |
27 |
28 |
29 |
30 |
31 |
32 |
118 |
119 |
120 |
121 |
122 |
127 |
128 |
129 |
137 |
138 |
139 |
140 |
249 |
250 |
--------------------------------------------------------------------------------
/2024/mistplay/mistplay-talk.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "5ab60f84-39b3-4bdd-ae83-6527acb315e5",
6 | "metadata": {},
7 | "source": [
8 | "# GenAI and LlamaIndex ◦ March 6 2024 ◦ Mistplay"
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": 1,
14 | "id": "2276edc2-74ce-4d85-bdaa-0607c74fc981",
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "##################################################################\n",
19 | "# Venue: Mistplay Offsite\n",
20 | "# Talk: GenAI and LlamaIndex\n",
21 | "# Speaker: Andrei Fajardo\n",
22 | "##################################################################"
23 | ]
24 | },
25 | {
26 | "cell_type": "markdown",
27 | "id": "3bee89b8-a04d-4326-9392-b9e7e1bcb8af",
28 | "metadata": {},
29 | "source": [
30 | ""
31 | ]
32 | },
33 | {
34 | "cell_type": "markdown",
35 | "id": "e4d38b38-ea48-4012-81ae-84e1d1f40a69",
36 | "metadata": {},
37 | "source": [
38 | ""
39 | ]
40 | },
41 | {
42 | "cell_type": "markdown",
43 | "id": "34d1f8e7-f978-4f19-bdfb-37c0d235b5bf",
44 | "metadata": {},
45 | "source": [
46 | "#### Notebook Setup & Dependency Installation"
47 | ]
48 | },
49 | {
50 | "cell_type": "code",
51 | "execution_count": 1,
52 | "id": "f227a52a-a147-4e8f-b7d3-e03f983fd5f1",
53 | "metadata": {
54 | "scrolled": true
55 | },
56 | "outputs": [
57 | {
58 | "name": "stdout",
59 | "output_type": "stream",
60 | "text": [
61 | "\n",
62 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
63 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.11 -m pip install --upgrade pip\u001b[0m\n",
64 | "Note: you may need to restart the kernel to use updated packages.\n"
65 | ]
66 | }
67 | ],
68 | "source": [
69 | "%pip install llama-index-vector-stores-qdrant llama-parse -q"
70 | ]
71 | },
72 | {
73 | "cell_type": "code",
74 | "execution_count": 1,
75 | "id": "7bc383fc-19b2-47b5-af61-e83210ea9c37",
76 | "metadata": {},
77 | "outputs": [],
78 | "source": [
79 | "import nest_asyncio\n",
80 | "\n",
81 | "nest_asyncio.apply()"
82 | ]
83 | },
84 | {
85 | "cell_type": "markdown",
86 | "id": "275c00f1-e358-498a-88c3-8e810a5a2546",
87 | "metadata": {},
88 | "source": [
89 | "## Motivation\n",
90 | "\n",
91 | "\n"
92 | ]
93 | },
94 | {
95 | "cell_type": "code",
96 | "execution_count": 39,
97 | "id": "25d4ce76-8eea-44cb-aa99-94844dfed9c7",
98 | "metadata": {},
99 | "outputs": [],
100 | "source": [
101 | "# query an LLM and ask it about Mistplay\n",
102 | "from llama_index.llms.openai import OpenAI\n",
103 | "\n",
104 | "llm = OpenAI(model=\"gpt-4\")\n",
105 | "response = llm.complete(\"What does Mistplay do?\")"
106 | ]
107 | },
108 | {
109 | "cell_type": "code",
110 | "execution_count": 40,
111 | "id": "c3f18489-4f25-40ce-86e9-697ddea7d6c6",
112 | "metadata": {},
113 | "outputs": [
114 | {
115 | "name": "stdout",
116 | "output_type": "stream",
117 | "text": [
118 | "Mistplay is a loyalty platform for mobile gamers. Users earn points for playing new games and can exchange them for gift cards or in-game purchases. It also allows users to discover and play new games, participate in contests, and connect with other gamers.\n"
119 | ]
120 | }
121 | ],
122 | "source": [
123 | "print(response)"
124 | ]
125 | },
126 | {
127 | "cell_type": "code",
128 | "execution_count": 41,
129 | "id": "80b2f9c3-1e07-4178-9e7e-220355725955",
130 | "metadata": {},
131 | "outputs": [],
132 | "source": [
133 | "response = llm.complete(\"What is the Mistplay Loyalty Index?\")"
134 | ]
135 | },
136 | {
137 | "cell_type": "code",
138 | "execution_count": 42,
139 | "id": "6a1c6424-cf2f-4ee1-b601-674b03354e35",
140 | "metadata": {},
141 | "outputs": [
142 | {
143 | "name": "stdout",
144 | "output_type": "stream",
145 | "text": [
146 | "The Mistplay Loyalty Index is a system used by the mobile gaming platform, Mistplay, to measure a player's engagement and loyalty to a particular game. The index is calculated based on various factors such as the amount of time spent playing the game, the frequency of play, and the player's in-game progress. The higher the index, the more rewards the player can earn on the Mistplay platform.\n"
147 | ]
148 | }
149 | ],
150 | "source": [
151 | "print(response)"
152 | ]
153 | },
154 | {
155 | "cell_type": "code",
156 | "execution_count": 43,
157 | "id": "632e1dec-045b-42fb-b325-c25e1ec4bb75",
158 | "metadata": {},
159 | "outputs": [],
160 | "source": [
161 | "response = llm.complete(\"What are the key survey insights from Mistplay's 2023 Mobile Gaming Loyalty Report?\")"
162 | ]
163 | },
164 | {
165 | "cell_type": "code",
166 | "execution_count": 44,
167 | "id": "53efac56-7966-407f-a298-bc2e0e14518e",
168 | "metadata": {},
169 | "outputs": [
170 | {
171 | "name": "stdout",
172 | "output_type": "stream",
173 | "text": [
174 | "As an AI, I can't provide real-time data or future predictions. As of my last update, it's still 2021, and I don't have access to Mistplay's 2023 Mobile Gaming Loyalty Report. I recommend checking Mistplay's official website or contacting them directly for the most accurate and up-to-date information.\n"
175 | ]
176 | }
177 | ],
178 | "source": [
179 | "print(response)"
180 | ]
181 | },
182 | {
183 | "cell_type": "markdown",
184 | "id": "04a0ef8d-d55c-4b64-887b-18d343503a76",
185 | "metadata": {},
186 | "source": [
187 | "## Basic RAG in 3 Steps\n",
188 | "\n",
189 | "\n",
190 | "\n",
191 | "\n",
192 | "1. Build external knowledge (i.e., uploading updated data sources)\n",
193 | "2. Retrieve\n",
194 | "3. Augment and Generate"
195 | ]
196 | },
197 | {
198 | "cell_type": "markdown",
199 | "id": "598a5257-20ae-468e-85d6-d4e8c46b8cb5",
200 | "metadata": {},
201 | "source": [
202 | "## 1. Build External Knowledge\n",
203 | "\n",
204 | ""
205 | ]
206 | },
207 | {
208 | "cell_type": "code",
209 | "execution_count": 45,
210 | "id": "a2963f90-9da5-4a0d-8dbe-f16fcb8627a3",
211 | "metadata": {},
212 | "outputs": [],
213 | "source": [
214 | "\"\"\"Load the data.\n",
215 | "\n",
216 | "With llama-index, before any transformations are applied,\n",
217 | "data is loaded in the `Document` abstraction, which is\n",
218 | "a container that holds the text of the document.\n",
219 | "\"\"\"\n",
220 | "\n",
221 | "from llama_index.core import SimpleDirectoryReader\n",
222 | "\n",
223 | "loader = SimpleDirectoryReader(input_dir=\"./data\")\n",
224 | "documents = loader.load_data()"
225 | ]
226 | },
227 | {
228 | "cell_type": "code",
229 | "execution_count": 46,
230 | "id": "da321e2c-8428-4c04-abf2-b204416e816f",
231 | "metadata": {},
232 | "outputs": [
233 | {
234 | "data": {
235 | "text/plain": [
236 | "\"2023\\u2028\\nMobile Gaming \\u2028\\nLoyalty Report\\u2028\\nDiscover what drives engagement and fosters \\nstrong loyalty among today's mobile gamers\\n\""
237 | ]
238 | },
239 | "execution_count": 46,
240 | "metadata": {},
241 | "output_type": "execute_result"
242 | }
243 | ],
244 | "source": [
245 | "# if you want to see what the text looks like\n",
246 | "documents[0].text[:1000]"
247 | ]
248 | },
249 | {
250 | "cell_type": "code",
251 | "execution_count": 47,
252 | "id": "4801e74a-8c52-45c4-967d-7a1a94f54ad3",
253 | "metadata": {},
254 | "outputs": [],
255 | "source": [
256 | "\"\"\"Chunk, Encode, and Store into a Vector Store.\n",
257 | "\n",
258 | "To streamline the process, we can make use of the IngestionPipeline\n",
259 | "class that will apply your specified transformations to the\n",
260 | "Document's.\n",
261 | "\"\"\"\n",
262 | "\n",
263 | "from llama_index.core.ingestion import IngestionPipeline\n",
264 | "from llama_index.core.node_parser import SentenceSplitter\n",
265 | "from llama_index.embeddings.openai import OpenAIEmbedding\n",
266 | "from llama_index.vector_stores.qdrant import QdrantVectorStore\n",
267 | "import qdrant_client\n",
268 | "\n",
269 | "client = qdrant_client.QdrantClient(location=\":memory:\")\n",
270 | "vector_store = QdrantVectorStore(client=client, collection_name=\"test_store\")\n",
271 | "\n",
272 | "pipeline = IngestionPipeline(\n",
273 | " transformations = [\n",
274 | " SentenceSplitter(),\n",
275 | " OpenAIEmbedding(),\n",
276 | " ],\n",
277 | " vector_store=vector_store,\n",
278 | ")\n",
279 | "_nodes = pipeline.run(documents=documents, num_workers=4)"
280 | ]
281 | },
282 | {
283 | "cell_type": "code",
284 | "execution_count": 48,
285 | "id": "02afea25-098b-49c7-a965-21c7576757af",
286 | "metadata": {},
287 | "outputs": [
288 | {
289 | "data": {
290 | "text/plain": [
291 | "\"2023\\u2028\\nMobile Gaming \\u2028\\nLoyalty Report\\u2028\\nDiscover what drives engagement and fosters \\nstrong loyalty among today's mobile gamers\""
292 | ]
293 | },
294 | "execution_count": 48,
295 | "metadata": {},
296 | "output_type": "execute_result"
297 | }
298 | ],
299 | "source": [
300 | "# if you want to see the nodes\n",
301 | "# len(_nodes)\n",
302 | "_nodes[0].text"
303 | ]
304 | },
305 | {
306 | "cell_type": "code",
307 | "execution_count": 49,
308 | "id": "44cd8a86-089d-4329-9484-35b98b3a26f9",
309 | "metadata": {},
310 | "outputs": [],
311 | "source": [
312 | "\"\"\"Create a llama-index... wait for it... Index.\n",
313 | "\n",
314 | "After uploading your encoded documents into your vector\n",
315 | "store of choice, you can connect to it with a VectorStoreIndex\n",
316 | "which then gives you access to all of the llama-index functionality.\n",
317 | "\"\"\"\n",
318 | "\n",
319 | "from llama_index.core import VectorStoreIndex\n",
320 | "\n",
321 | "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)"
322 | ]
323 | },
324 | {
325 | "cell_type": "markdown",
326 | "id": "286b1827-7547-49c6-aba3-82f08d6d86b8",
327 | "metadata": {},
328 | "source": [
329 | "## 2. Retrieve Against A Query\n",
330 | "\n",
331 | ""
332 | ]
333 | },
334 | {
335 | "cell_type": "code",
336 | "execution_count": 50,
337 | "id": "49f86af1-db08-4641-89ad-d60abd04e6b3",
338 | "metadata": {},
339 | "outputs": [],
340 | "source": [
341 | "\"\"\"Retrieve relevant documents against a query.\n",
342 | "\n",
343 | "With our Index ready, we can now query it to\n",
344 | "retrieve the most relevant document chunks.\n",
345 | "\"\"\"\n",
346 | "\n",
347 | "retriever = index.as_retriever(similarity_top_k=2)\n",
348 | "retrieved_nodes = retriever.retrieve(\"What are the key survey insights from Mistplay's 2023 Mobile Gaming Loyalty Report?\")"
349 | ]
350 | },
351 | {
352 | "cell_type": "code",
353 | "execution_count": 51,
354 | "id": "05f9ce3b-a4e3-4862-b58c-2d9fba1f9abc",
355 | "metadata": {},
356 | "outputs": [
357 | {
358 | "name": "stdout",
359 | "output_type": "stream",
360 | "text": [
361 | "3 % ) . A s p l a y e r s p r o g r e s s , t h e y \n",
362 | "a l s o e n j o y s e e i n g t h e n e w e s t f e a t u r e s a n d u p d a t e s a d d e d t o t h e g a m e , w i t h 3 7 . 7 % o f p l a y e r s c i t i n g i t a s a r e a s o n f o r r e t u r n i n g . B u i l d i n g c o r e g a m e p l a y l o o p s t h a t p r o v i d e m e a n i n g f u l a n d v i s i b l e p r o g r e s s i o n – c o m p l e t i n g l e v e l s , u n l o c k i n g n e w \n"
363 | ]
364 | }
365 | ],
366 | "source": [
367 | "# to view the retrieved node\n",
368 | "print(retrieved_nodes[0].text[:500])"
369 | ]
370 | },
371 | {
372 | "cell_type": "markdown",
373 | "id": "978ae2c5-8c2a-41c7-a2eb-85a5562f2db5",
374 | "metadata": {},
375 | "source": [
376 | "## 3. Generate Final Response\n",
377 | "\n",
378 | ""
379 | ]
380 | },
381 | {
382 | "cell_type": "code",
383 | "execution_count": 52,
384 | "id": "ef33c349-eed4-4e35-9b5d-9473adf2ce01",
385 | "metadata": {},
386 | "outputs": [],
387 | "source": [
388 | "\"\"\"Context-Augemented Generation.\n",
389 | "\n",
390 | "With our Index ready, we can create a QueryEngine\n",
391 | "that handles the retrieval and context augmentation\n",
392 | "in order to get the final response.\n",
393 | "\"\"\"\n",
394 | "\n",
395 | "query_engine = index.as_query_engine()"
396 | ]
397 | },
398 | {
399 | "cell_type": "code",
400 | "execution_count": 16,
401 | "id": "4139c48a-ece8-4244-b4eb-7cff74cb1325",
402 | "metadata": {},
403 | "outputs": [
404 | {
405 | "name": "stdout",
406 | "output_type": "stream",
407 | "text": [
408 | "Context information is below.\n",
409 | "---------------------\n",
410 | "{context_str}\n",
411 | "---------------------\n",
412 | "Given the context information and not prior knowledge, answer the query.\n",
413 | "Query: {query_str}\n",
414 | "Answer: \n"
415 | ]
416 | }
417 | ],
418 | "source": [
419 | "# to inspect the default prompt being used\n",
420 | "print(query_engine.\n",
421 | " get_prompts()[\"response_synthesizer:text_qa_template\"]\n",
422 | " .default_template.template\n",
423 | ")"
424 | ]
425 | },
426 | {
427 | "cell_type": "code",
428 | "execution_count": 53,
429 | "id": "6179639d-af96-4a09-b440-b47ad599a26f",
430 | "metadata": {},
431 | "outputs": [
432 | {
433 | "name": "stdout",
434 | "output_type": "stream",
435 | "text": [
436 | "The key survey insights from Mistplay's 2023 Mobile Gaming Loyalty Report include motivations for continuing to play favorite games, such as the desire to keep progressing and reach new milestones, not wanting to abandon progress and achievements, enjoying new features and content updates, enjoying the competitiveness of the game and mastering skill levels, spending money in the games, believing the game is the best in its genre, playing because friends and/or family also play, and participating in the game's community. Additionally, the report highlights that mobile game advertising is the most successful method for players to discover new games, with nearly half relying on ads as a primary method, followed by app store exploration and direct recommendations from friends or family.\n"
437 | ]
438 | }
439 | ],
440 | "source": [
441 | "response = query_engine.query(\"What are the key survey insights from Mistplay's 2023 Mobile Gaming Loyalty Report?\")\n",
442 | "print(response)"
443 | ]
444 | },
445 | {
446 | "cell_type": "markdown",
447 | "id": "a050bb7f-266c-46c7-8773-eccb909e5fd6",
448 | "metadata": {},
449 | "source": [
450 | "## Beyond Basic RAG: Improved PDF Parsing with LlamaParse"
451 | ]
452 | },
453 | {
454 | "cell_type": "markdown",
455 | "id": "a4a08bc5-1ff9-42f5-91a3-f77f3a529ff1",
456 | "metadata": {},
457 | "source": [
458 | "### The default pdf reader (PyPDF), like many out-of-the box pdf parsers struggle on complex PDF docs."
459 | ]
460 | },
461 | {
462 | "cell_type": "code",
463 | "execution_count": 28,
464 | "id": "32b6d6bd-f04a-400a-99ac-b9c0308c7554",
465 | "metadata": {},
466 | "outputs": [
467 | {
468 | "name": "stdout",
469 | "output_type": "stream",
470 | "text": [
471 | "The three listed key index insights from Mistplay's 2023 Mobile Gaming Loyalty Report are Loyalty Index, Loyalty Ratings, and Monetization Metrics.\n"
472 | ]
473 | }
474 | ],
475 | "source": [
476 | "response = query_engine.query(\"What are the three listed key index insights from Mistplay's 2023 Mobile Gaming Loyalty Report?\")\n",
477 | "print(response)"
478 | ]
479 | },
480 | {
481 | "cell_type": "code",
482 | "execution_count": 36,
483 | "id": "40e70776-611d-4b52-bd89-3f233cdaa9c5",
484 | "metadata": {},
485 | "outputs": [
486 | {
487 | "name": "stdout",
488 | "output_type": "stream",
489 | "text": [
490 | "RPG has the highest loyalty index value, followed by Strategy and Simulation in the top 3 categories of games according to Mistplay's 2023 Mobile Gaming Loyalty Report.\n"
491 | ]
492 | }
493 | ],
494 | "source": [
495 | "response = query_engine.query(\"What are the top 3 categories of games which get the most loyalty (and list their index values) according to Mistplay's 2023 Mobile Gaming Loyalty Report?\")\n",
496 | "print(response)"
497 | ]
498 | },
499 | {
500 | "cell_type": "markdown",
501 | "id": "3895970c-dae2-4ae3-b48d-72da7617fcd3",
502 | "metadata": {},
503 | "source": [
504 | "### Improved PDF Parsing using LlamaParse"
505 | ]
506 | },
507 | {
508 | "cell_type": "code",
509 | "execution_count": 23,
510 | "id": "ad81e2cf-90b6-49af-b36e-2f896abec0c8",
511 | "metadata": {},
512 | "outputs": [
513 | {
514 | "name": "stdout",
515 | "output_type": "stream",
516 | "text": [
517 | "Started parsing the file under job_id e256d886-1db2-449a-aaca-5d64200286ba\n",
518 | "....................."
519 | ]
520 | }
521 | ],
522 | "source": [
523 | "from llama_parse import LlamaParse\n",
524 | "\n",
525 | "parser = LlamaParse(result_type=\"markdown\")\n",
526 | "documents = parser.load_data(file_path=\"./data/2023 Mobile Gaming Loyalty Report by Mistplay-3.pdf\")"
527 | ]
528 | },
529 | {
530 | "cell_type": "code",
531 | "execution_count": 24,
532 | "id": "efaed5ae-9c87-4670-b571-9dff91b06cea",
533 | "metadata": {},
534 | "outputs": [],
535 | "source": [
536 | "llama_parse_index = VectorStoreIndex.from_documents(documents)\n",
537 | "llama_parse_query_engine = llama_parse_index.as_query_engine()"
538 | ]
539 | },
540 | {
541 | "cell_type": "code",
542 | "execution_count": 54,
543 | "id": "3e54ff4a-22d8-45b4-b27c-84387ff7b4bb",
544 | "metadata": {},
545 | "outputs": [
546 | {
547 | "name": "stdout",
548 | "output_type": "stream",
549 | "text": [
550 | "Players are motivated by play-and-earn. Word of mouth is a top-3 acquisition source. Players play multiple games simultaneously but only spend money on a few. Additionally, players are interested in games where they can play and earn tangible rewards, with a significant percentage willing to switch from their current favorite game for this feature.\n"
551 | ]
552 | }
553 | ],
554 | "source": [
555 | "response = llama_parse_query_engine.query(\"What are the key survey insights from Mistplay's 2023 Mobile Gaming Loyalty Report?\")\n",
556 | "print(response)"
557 | ]
558 | },
559 | {
560 | "cell_type": "code",
561 | "execution_count": 29,
562 | "id": "328ee422-cd6d-4d5d-bfe9-da5e878f85a5",
563 | "metadata": {},
564 | "outputs": [
565 | {
566 | "name": "stdout",
567 | "output_type": "stream",
568 | "text": [
569 | "RPG games garner the most loyalty. Strategy games drive average sessions per user. Lifestyle games see the highest repeat purchases.\n"
570 | ]
571 | }
572 | ],
573 | "source": [
574 | "response = llama_parse_query_engine.query(\"What are the three listed key index insights from Mistplay's 2023 Mobile Gaming Loyalty Report?\")\n",
575 | "print(response)"
576 | ]
577 | },
578 | {
579 | "cell_type": "code",
580 | "execution_count": 35,
581 | "id": "516c2d0f-c621-40fc-b7a7-17ffc28d2713",
582 | "metadata": {},
583 | "outputs": [
584 | {
585 | "name": "stdout",
586 | "output_type": "stream",
587 | "text": [
588 | "RPG games have the highest loyalty rating of 75, followed by Strategy games with a loyalty rating of 70, and Simulation games with a loyalty rating of 67.\n"
589 | ]
590 | }
591 | ],
592 | "source": [
593 | "response = llama_parse_query_engine.query(\"What are the top 3 categories of games which get the most loyalty (and list their index values) according to Mistplay's 2023 Mobile Gaming Loyalty Report?\")\n",
594 | "print(response)"
595 | ]
596 | },
597 | {
598 | "cell_type": "markdown",
599 | "id": "dae63946-be38-4807-af2a-8113661a806b",
600 | "metadata": {},
601 | "source": [
602 | "## In Summary\n",
603 | "\n",
604 | "- LLMs as powerful as they are, don't perform too well with knowledge-intensive tasks (domain specific, updated data, long-tail)\n",
605 | "- Context augmentation has been shown (in a few studies) to outperform LLMs without augmentation\n",
606 | "- In this notebook, we showed one such example that follows that pattern."
607 | ]
608 | },
609 | {
610 | "cell_type": "markdown",
611 | "id": "fc857227-3fed-4bb6-a062-99ea3c55e294",
612 | "metadata": {},
613 | "source": [
614 | "# LlamaIndex Has More To Offer\n",
615 | "\n",
616 | "- Data infrastructure that enables production-grade, advanced RAG systems\n",
617 | "- Agentic solutions\n",
618 | "- Newly released: `llama-index-networks`\n",
619 | "- Enterprise offerings (alpha):\n",
620 | " - LlamaParse (proprietary complex PDF parser) and\n",
621 | " - LlamaCloud"
622 | ]
623 | },
624 | {
625 | "cell_type": "markdown",
626 | "id": "17c1c027-be8b-48f4-87ee-06f3e2c71797",
627 | "metadata": {},
628 | "source": [
629 | "### Useful links\n",
630 | "\n",
631 | "[website](https://www.llamaindex.ai/) ◦ [llamahub](https://llamahub.ai) ◦ [llamaparse](https://github.com/run-llama/llama_parse) ◦ [github](https://github.com/run-llama/llama_index) ◦ [medium](https://medium.com/@llama_index) ◦ [rag-bootcamp-poster](https://d3ddy8balm3goa.cloudfront.net/rag-bootcamp-vector/final_poster.excalidraw.svg)"
632 | ]
633 | },
634 | {
635 | "cell_type": "code",
636 | "execution_count": null,
637 | "id": "4876f95e-0a1c-4fcf-b155-c8ed688f6649",
638 | "metadata": {},
639 | "outputs": [],
640 | "source": []
641 | }
642 | ],
643 | "metadata": {
644 | "kernelspec": {
645 | "display_name": "mistplay",
646 | "language": "python",
647 | "name": "mistplay"
648 | },
649 | "language_info": {
650 | "codemirror_mode": {
651 | "name": "ipython",
652 | "version": 3
653 | },
654 | "file_extension": ".py",
655 | "mimetype": "text/x-python",
656 | "name": "python",
657 | "nbconvert_exporter": "python",
658 | "pygments_lexer": "ipython3",
659 | "version": "3.11.3"
660 | }
661 | },
662 | "nbformat": 4,
663 | "nbformat_minor": 5
664 | }
665 |
--------------------------------------------------------------------------------
/2024/mistplay/network_query_engine.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "27beb3c1-4d05-4645-85e5-60c28bdaaae2",
6 | "metadata": {},
7 | "source": [
8 | "# NetworkQueryEngine Demo"
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": 1,
14 | "id": "912fbf7c-e77d-4cfb-8e22-bba449d422a8",
15 | "metadata": {},
16 | "outputs": [
17 | {
18 | "name": "stdout",
19 | "output_type": "stream",
20 | "text": [
21 | "\u001b[33mWARNING: pydantic 2.6.3 does not provide the extra 'dotenv'\u001b[0m\u001b[33m\n",
22 | "\u001b[0m\n",
23 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
24 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.11 -m pip install --upgrade pip\u001b[0m\n",
25 | "Note: you may need to restart the kernel to use updated packages.\n"
26 | ]
27 | }
28 | ],
29 | "source": [
30 | "%pip install llama-index-networks -q"
31 | ]
32 | },
33 | {
34 | "cell_type": "code",
35 | "execution_count": 2,
36 | "id": "ff743622-6887-4121-8049-44110b366f06",
37 | "metadata": {},
38 | "outputs": [],
39 | "source": [
40 | "import os\n",
41 | "from llama_index.networks import (\n",
42 | " NetworkQueryEngine,\n",
43 | " ContributorClient,\n",
44 | ")\n",
45 | "from llama_index.llms.openai import OpenAI"
46 | ]
47 | },
48 | {
49 | "cell_type": "code",
50 | "execution_count": 3,
51 | "id": "ee0673fd-2bcc-4a45-8caa-a2714377ac5c",
52 | "metadata": {},
53 | "outputs": [],
54 | "source": [
55 | "import nest_asyncio\n",
56 | "\n",
57 | "nest_asyncio.apply()"
58 | ]
59 | },
60 | {
61 | "cell_type": "code",
62 | "execution_count": 4,
63 | "id": "c1c19958-cb5d-49b4-af8a-76265f108286",
64 | "metadata": {},
65 | "outputs": [],
66 | "source": [
67 | "# setup ContributorClients to connect to ContributorServices\n",
68 | "contributors = [\n",
69 | " ContributorClient.from_config_file(\n",
70 | " env_file=f\"./client-env-files/.env.contributor_{ix}.client\"\n",
71 | " )\n",
72 | " for ix in range(1, 4)\n",
73 | "]"
74 | ]
75 | },
76 | {
77 | "cell_type": "code",
78 | "execution_count": 5,
79 | "id": "ce075783-b9bc-4dbb-b982-e6c2c47a8a40",
80 | "metadata": {},
81 | "outputs": [],
82 | "source": [
83 | "# build NetworkRAG\n",
84 | "api_key = os.environ.get(\"OPENAI_API_KEY\")\n",
85 | "llm = OpenAI(api_key=api_key)\n",
86 | "network_query_engine = NetworkQueryEngine.from_args(contributors=contributors, llm=llm)"
87 | ]
88 | },
89 | {
90 | "cell_type": "code",
91 | "execution_count": 6,
92 | "id": "05350567-a369-4274-9574-cecb194f1d7d",
93 | "metadata": {},
94 | "outputs": [],
95 | "source": [
96 | "# knowledge from contributor 1\n",
97 | "response = await network_query_engine.aquery(\"Who is Paul?\")"
98 | ]
99 | },
100 | {
101 | "cell_type": "code",
102 | "execution_count": 7,
103 | "id": "9ccb68f2-2c65-4074-bb7f-5c411ab8ef5c",
104 | "metadata": {},
105 | "outputs": [
106 | {
107 | "data": {
108 | "text/plain": [
109 | "\"Paul is a person who has been involved in various ventures such as painting, starting companies like Viaweb, and working on web applications. He also delved into writing essays and realized the potential of publishing them online. Paul's experiences led him to focus on working on things that may not be prestigious but hold real value and align with his motives.\""
110 | ]
111 | },
112 | "execution_count": 7,
113 | "metadata": {},
114 | "output_type": "execute_result"
115 | }
116 | ],
117 | "source": [
118 | "response.response"
119 | ]
120 | },
121 | {
122 | "cell_type": "code",
123 | "execution_count": 8,
124 | "id": "7f28db53-a96d-431a-aee9-bf16b236f850",
125 | "metadata": {},
126 | "outputs": [],
127 | "source": [
128 | "# knowledge from contributor 2\n",
129 | "response = await network_query_engine.aquery(\"What is DoRA?\")"
130 | ]
131 | },
132 | {
133 | "cell_type": "code",
134 | "execution_count": 9,
135 | "id": "03144a51-21ec-4667-858e-81baaea97507",
136 | "metadata": {},
137 | "outputs": [
138 | {
139 | "data": {
140 | "text/plain": [
141 | "'DoRA is a method that enhances the learning capacity of LoRA by introducing incremental directional updates that can be adapted by different LoRA variants. It can be combined with other LoRA variants like VeRA, which reduces trainable parameters significantly while maintaining accuracy. DoRA consistently outperforms LoRA across various rank settings, showcasing its ability to achieve better accuracy with fewer trainable parameters.'"
142 | ]
143 | },
144 | "execution_count": 9,
145 | "metadata": {},
146 | "output_type": "execute_result"
147 | }
148 | ],
149 | "source": [
150 | "response.response"
151 | ]
152 | },
153 | {
154 | "cell_type": "code",
155 | "execution_count": 10,
156 | "id": "bd5849c6-6ebf-4f69-bbd2-d874e2ee80d9",
157 | "metadata": {},
158 | "outputs": [],
159 | "source": [
160 | "# knowledge from contributor 3\n",
161 | "response = await network_query_engine.aquery(\"Summarize the history of New York City\")"
162 | ]
163 | },
164 | {
165 | "cell_type": "code",
166 | "execution_count": 11,
167 | "id": "de285ade-451f-407d-a308-685da6b5fcb4",
168 | "metadata": {},
169 | "outputs": [
170 | {
171 | "data": {
172 | "text/plain": [
173 | "\"New York City's history began with Dutch settlement in the early 17th century, evolving into a major port city and immigration hub. It played a crucial role in the American Revolution and briefly served as the national capital. The city's strategic location and economic growth in the 19th and 20th centuries, fueled by European immigrants, solidified its position as a major urban center globally. Today, New York City is renowned for its finance, culture, and diverse population, reflecting its enduring legacy as a city of opportunity and innovation.\""
174 | ]
175 | },
176 | "execution_count": 11,
177 | "metadata": {},
178 | "output_type": "execute_result"
179 | }
180 | ],
181 | "source": [
182 | "response.response"
183 | ]
184 | },
185 | {
186 | "cell_type": "code",
187 | "execution_count": null,
188 | "id": "0f84aacb-96ef-493c-b9c8-d1c313a3d093",
189 | "metadata": {},
190 | "outputs": [],
191 | "source": []
192 | }
193 | ],
194 | "metadata": {
195 | "kernelspec": {
196 | "display_name": "mistplay",
197 | "language": "python",
198 | "name": "mistplay"
199 | },
200 | "language_info": {
201 | "codemirror_mode": {
202 | "name": "ipython",
203 | "version": 3
204 | },
205 | "file_extension": ".py",
206 | "mimetype": "text/x-python",
207 | "name": "python",
208 | "nbconvert_exporter": "python",
209 | "pygments_lexer": "ipython3",
210 | "version": "3.11.3"
211 | }
212 | },
213 | "nbformat": 4,
214 | "nbformat_minor": 5
215 | }
216 |
--------------------------------------------------------------------------------
/2024/mlops/mlops-rag-bootcamp.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 6,
6 | "id": "61041f0b-e704-4b1b-a582-30872db4c1e1",
7 | "metadata": {},
8 | "outputs": [],
9 | "source": [
10 | "##################################################################\n",
11 | "# Venue: MLOps Learners RAG Workshop\n",
12 | "# Talk: Intro to RAG\n",
13 | "# Speaker: Andrei Fajardo\n",
14 | "##################################################################"
15 | ]
16 | },
17 | {
18 | "cell_type": "markdown",
19 | "id": "569ab52d-4652-4829-8188-295e3d91bb8f",
20 | "metadata": {},
21 | "source": [
22 | "\n",
23 | ""
24 | ]
25 | },
26 | {
27 | "cell_type": "markdown",
28 | "id": "837492d6-106f-49dc-b610-f7052181d17f",
29 | "metadata": {},
30 | "source": [
31 | "\n",
32 | ""
33 | ]
34 | },
35 | {
36 | "cell_type": "markdown",
37 | "id": "34d1f8e7-f978-4f19-bdfb-37c0d235b5bf",
38 | "metadata": {},
39 | "source": [
40 | "#### Notebook Setup & Dependency Installation"
41 | ]
42 | },
43 | {
44 | "cell_type": "code",
45 | "execution_count": null,
46 | "id": "f227a52a-a147-4e8f-b7d3-e03f983fd5f1",
47 | "metadata": {
48 | "scrolled": true
49 | },
50 | "outputs": [],
51 | "source": [
52 | "%pip install llama-index llama-parse llama-index-vector-stores-qdrant -q"
53 | ]
54 | },
55 | {
56 | "cell_type": "code",
57 | "execution_count": 1,
58 | "id": "7bc383fc-19b2-47b5-af61-e83210ea9c37",
59 | "metadata": {},
60 | "outputs": [],
61 | "source": [
62 | "import nest_asyncio\n",
63 | "\n",
64 | "nest_asyncio.apply()"
65 | ]
66 | },
67 | {
68 | "cell_type": "code",
69 | "execution_count": 2,
70 | "id": "d56aba9c-230b-464d-909c-de17d1bc0aa6",
71 | "metadata": {},
72 | "outputs": [
73 | {
74 | "name": "stdout",
75 | "output_type": "stream",
76 | "text": [
77 | "mkdir: data: File exists\n",
78 | "--2024-04-04 11:34:26-- https://vectorinstitute.ai/wp-content/uploads/2024/02/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\n",
79 | "Resolving vectorinstitute.ai (vectorinstitute.ai)... 162.159.134.42\n",
80 | "Connecting to vectorinstitute.ai (vectorinstitute.ai)|162.159.134.42|:443... connected.\n",
81 | "HTTP request sent, awaiting response... 200 OK\n",
82 | "Length: 5305903 (5.1M) [application/pdf]\n",
83 | "Saving to: ‘./data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf’\n",
84 | "\n",
85 | "./data/Vector-Annua 100%[===================>] 5.06M 14.2MB/s in 0.4s \n",
86 | "\n",
87 | "2024-04-04 11:34:27 (14.2 MB/s) - ‘./data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf’ saved [5305903/5305903]\n",
88 | "\n"
89 | ]
90 | }
91 | ],
92 | "source": [
93 | "!mkdir data\n",
94 | "!wget \"https://vectorinstitute.ai/wp-content/uploads/2024/02/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\" -O \"./data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\""
95 | ]
96 | },
97 | {
98 | "cell_type": "markdown",
99 | "id": "31ecf9b4-e9c8-43e6-927d-4b85a34bcf0d",
100 | "metadata": {},
101 | "source": [
102 | "\n",
103 | ""
104 | ]
105 | },
106 | {
107 | "cell_type": "markdown",
108 | "id": "275c00f1-e358-498a-88c3-8e810a5a2546",
109 | "metadata": {},
110 | "source": [
111 | "\n",
112 | ""
113 | ]
114 | },
115 | {
116 | "cell_type": "code",
117 | "execution_count": 3,
118 | "id": "25d4ce76-8eea-44cb-aa99-94844dfed9c7",
119 | "metadata": {},
120 | "outputs": [],
121 | "source": [
122 | "# query an LLM and ask it about Mistplay\n",
123 | "from llama_index.llms.openai import OpenAI\n",
124 | "\n",
125 | "llm = OpenAI(model=\"gpt-4-turbo-preview\")\n",
126 | "\n",
127 | "response = llm.complete(\n",
128 | " \"What is Vector Institute all about?\"\n",
129 | ")"
130 | ]
131 | },
132 | {
133 | "cell_type": "code",
134 | "execution_count": 4,
135 | "id": "c3f18489-4f25-40ce-86e9-697ddea7d6c6",
136 | "metadata": {},
137 | "outputs": [
138 | {
139 | "name": "stdout",
140 | "output_type": "stream",
141 | "text": [
142 | "The Vector Institute is a research institution in Toronto, Canada, dedicated to the field of artificial intelligence (AI), with a particular focus on machine learning and deep learning. It was established in March 2017 as part of Canada's strategy to lead in the AI domain, leveraging the country's significant contributions to the development of AI technologies.\n",
143 | "\n",
144 | "The institute's mission revolves around several key objectives:\n",
145 | "\n",
146 | "1. **Research Excellence**: Vector aims to attract, retain, and develop top talent in the AI field. It fosters an environment where leading researchers and industry experts can collaborate on cutting-edge projects, pushing the boundaries of what AI can achieve.\n",
147 | "\n",
148 | "2. **Industry Collaboration**: The institute works closely with companies across various sectors, helping them to adopt AI technologies to drive innovation and competitiveness. This includes providing guidance on AI strategy, development, and deployment.\n",
149 | "\n",
150 | "3. **Talent Development**: Recognizing the critical need for skilled professionals in the AI field, Vector offers educational programs and training opportunities. These initiatives are designed to equip students, professionals, and leaders with the knowledge and skills necessary to excel in the rapidly evolving AI landscape.\n",
151 | "\n",
152 | "4. **Thought Leadership**: Vector contributes to the global conversation on AI by sharing insights, research findings, and best practices. It aims to influence policy and decision-making, ensuring that AI development and deployment are guided by ethical considerations and societal needs.\n",
153 | "\n",
154 | "The Vector Institute is part of a broader Canadian AI ecosystem, which includes other leading institutions like the Montreal Institute for Learning Algorithms (MILA) and the Alberta Machine Intelligence Institute (Amii). Together, these organizations help to solidify Canada's position as a world leader in AI research and innovation.\n"
155 | ]
156 | }
157 | ],
158 | "source": [
159 | "print(response)"
160 | ]
161 | },
162 | {
163 | "cell_type": "code",
164 | "execution_count": 5,
165 | "id": "80b2f9c3-1e07-4178-9e7e-220355725955",
166 | "metadata": {},
167 | "outputs": [],
168 | "source": [
169 | "response = llm.complete(\n",
170 | " \"According to Vector Institute's Annual Report 2022-2023, \"\n",
171 | " \"how many AI jobs were created in Ontario?\"\n",
172 | ")"
173 | ]
174 | },
175 | {
176 | "cell_type": "code",
177 | "execution_count": 6,
178 | "id": "6a1c6424-cf2f-4ee1-b601-674b03354e35",
179 | "metadata": {},
180 | "outputs": [
181 | {
182 | "name": "stdout",
183 | "output_type": "stream",
184 | "text": [
185 | "As of my last update in April 2023, I don't have access to the specific details of the Vector Institute's Annual Report for 2022-2023. Therefore, I cannot provide the exact number of AI jobs created in Ontario as reported in that document. For the most accurate and up-to-date information, I recommend checking the Vector Institute's official website or contacting them directly. They may have published the report or related summaries that include the data you're interested in.\n"
186 | ]
187 | }
188 | ],
189 | "source": [
190 | "print(response)"
191 | ]
192 | },
193 | {
194 | "cell_type": "markdown",
195 | "id": "04a0ef8d-d55c-4b64-887b-18d343503a76",
196 | "metadata": {},
197 | "source": [
198 | "\n",
199 | "\n",
200 | "\n",
201 | "**Three Steps To Basic RAG:**\n",
202 | "1. Build external knowledge (i.e., uploading updated data sources)\n",
203 | "2. Retrieve\n",
204 | "3. Augment and Generate\n",
205 | "\n",
206 | ""
207 | ]
208 | },
209 | {
210 | "cell_type": "markdown",
211 | "id": "598a5257-20ae-468e-85d6-d4e8c46b8cb5",
212 | "metadata": {},
213 | "source": [
214 | "## 1. Build External Knowledge\n",
215 | "\n",
216 | ""
217 | ]
218 | },
219 | {
220 | "cell_type": "code",
221 | "execution_count": 7,
222 | "id": "a2963f90-9da5-4a0d-8dbe-f16fcb8627a3",
223 | "metadata": {},
224 | "outputs": [],
225 | "source": [
226 | "\"\"\"Load the data.\n",
227 | "\n",
228 | "With llama-index, before any transformations are applied,\n",
229 | "data is loaded in the `Document` abstraction, which is\n",
230 | "a container that holds the text of the document.\n",
231 | "\"\"\"\n",
232 | "\n",
233 | "from llama_index.core import SimpleDirectoryReader\n",
234 | "\n",
235 | "loader = SimpleDirectoryReader(input_dir=\"./data\")\n",
236 | "documents = loader.load_data()"
237 | ]
238 | },
239 | {
240 | "cell_type": "code",
241 | "execution_count": 8,
242 | "id": "da321e2c-8428-4c04-abf2-b204416e816f",
243 | "metadata": {},
244 | "outputs": [
245 | {
246 | "data": {
247 | "text/plain": [
248 | "'Vector Institute | Annual Report 2022-2023 2\\nTABLE OF CONTENTS\\nMESSAGE FROM THE BOARD CHAIR 3 \\nMESSAGE FROM THE PRESIDENT AND CEO 4\\nVECTOR’S VISION AND MISSION 5\\nONTARIO’S VIBRANT AI ECOSYSTEM 6\\nINDUSTRY INNOVATION 7\\nRESEARCH AND EDUCATION 1 4TALENT AND WORKFORCE DEVELOPMENT 24\\nHEALTH 3 1\\nAI ENGINEERING 36\\nTHOUGHT LEADERSHIP 39\\nFINANCIALS 42\\n'"
249 | ]
250 | },
251 | "execution_count": 8,
252 | "metadata": {},
253 | "output_type": "execute_result"
254 | }
255 | ],
256 | "source": [
257 | "# if you want to see what the text looks like\n",
258 | "documents[1].text"
259 | ]
260 | },
261 | {
262 | "cell_type": "code",
263 | "execution_count": 9,
264 | "id": "4801e74a-8c52-45c4-967d-7a1a94f54ad3",
265 | "metadata": {},
266 | "outputs": [],
267 | "source": [
268 | "\"\"\"Chunk, Encode, and Store into a Vector Store.\n",
269 | "\n",
270 | "To streamline the process, we can make use of the IngestionPipeline\n",
271 | "class that will apply your specified transformations to the\n",
272 | "Document's.\n",
273 | "\"\"\"\n",
274 | "\n",
275 | "from llama_index.core.ingestion import IngestionPipeline\n",
276 | "from llama_index.core.node_parser import SentenceSplitter\n",
277 | "from llama_index.embeddings.openai import OpenAIEmbedding\n",
278 | "from llama_index.vector_stores.qdrant import QdrantVectorStore\n",
279 | "import qdrant_client\n",
280 | "\n",
281 | "client = qdrant_client.QdrantClient(location=\":memory:\")\n",
282 | "vector_store = QdrantVectorStore(client=client, collection_name=\"test_store\")\n",
283 | "\n",
284 | "pipeline = IngestionPipeline(\n",
285 | " transformations = [\n",
286 | " SentenceSplitter(),\n",
287 | " OpenAIEmbedding(),\n",
288 | " ],\n",
289 | " vector_store=vector_store,\n",
290 | ")\n",
291 | "_nodes = pipeline.run(documents=documents, num_workers=4)"
292 | ]
293 | },
294 | {
295 | "cell_type": "code",
296 | "execution_count": 10,
297 | "id": "02afea25-098b-49c7-a965-21c7576757af",
298 | "metadata": {},
299 | "outputs": [
300 | {
301 | "data": {
302 | "text/plain": [
303 | "'Vector Institute | Annual Report 2022-2023 2\\nTABLE OF CONTENTS\\nMESSAGE FROM THE BOARD CHAIR 3 \\nMESSAGE FROM THE PRESIDENT AND CEO 4\\nVECTOR’S VISION AND MISSION 5\\nONTARIO’S VIBRANT AI ECOSYSTEM 6\\nINDUSTRY INNOVATION 7\\nRESEARCH AND EDUCATION 1 4TALENT AND WORKFORCE DEVELOPMENT 24\\nHEALTH 3 1\\nAI ENGINEERING 36\\nTHOUGHT LEADERSHIP 39\\nFINANCIALS 42'"
304 | ]
305 | },
306 | "execution_count": 10,
307 | "metadata": {},
308 | "output_type": "execute_result"
309 | }
310 | ],
311 | "source": [
312 | "# if you want to see the nodes\n",
313 | "# len(_nodes)\n",
314 | "_nodes[1].text"
315 | ]
316 | },
317 | {
318 | "cell_type": "code",
319 | "execution_count": 11,
320 | "id": "44cd8a86-089d-4329-9484-35b98b3a26f9",
321 | "metadata": {},
322 | "outputs": [],
323 | "source": [
324 | "\"\"\"Create a llama-index... wait for it... Index.\n",
325 | "\n",
326 | "After uploading your encoded documents into your vector\n",
327 | "store of choice, you can connect to it with a VectorStoreIndex\n",
328 | "which then gives you access to all of the llama-index functionality.\n",
329 | "\"\"\"\n",
330 | "\n",
331 | "from llama_index.core import VectorStoreIndex\n",
332 | "\n",
333 | "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)"
334 | ]
335 | },
336 | {
337 | "cell_type": "markdown",
338 | "id": "286b1827-7547-49c6-aba3-82f08d6d86b8",
339 | "metadata": {},
340 | "source": [
341 | "## 2. Retrieve Against A Query\n",
342 | "\n",
343 | ""
344 | ]
345 | },
346 | {
347 | "cell_type": "code",
348 | "execution_count": 13,
349 | "id": "49f86af1-db08-4641-89ad-d60abd04e6b3",
350 | "metadata": {},
351 | "outputs": [],
352 | "source": [
353 | "\"\"\"Retrieve relevant documents against a query.\n",
354 | "\n",
355 | "With our Index ready, we can now query it to\n",
356 | "retrieve the most relevant document chunks.\n",
357 | "\"\"\"\n",
358 | "\n",
359 | "retriever = index.as_retriever(similarity_top_k=2)\n",
360 | "retrieved_nodes = retriever.retrieve(\n",
361 | " \"According to Vector Institute's Annual Report 2022-2023, \"\n",
362 | " \"how many AI jobs were created in Ontario?\"\n",
363 | ")"
364 | ]
365 | },
366 | {
367 | "cell_type": "code",
368 | "execution_count": 14,
369 | "id": "06b04a79-2a28-4a21-b15b-4bb6c7b1a227",
370 | "metadata": {},
371 | "outputs": [
372 | {
373 | "data": {
374 | "text/plain": [
375 | "[NodeWithScore(node=TextNode(id_='af5b8613-9990-492f-8a0c-94ced4be3c61', embedding=None, metadata={'page_label': '6', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={: RelatedNodeInfo(node_id='47692fff-7f25-4472-846f-c3f8d30a16e5', node_type=, metadata={'page_label': '6', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='3bbac897823b34c5131c0df0923d9cc6125119814b241c192bb600a50328b432'), : RelatedNodeInfo(node_id='a84bb9b1-771a-40b1-b923-bb5c7d18f02b', node_type=, metadata={'page_label': '5', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='31dd177bdc920cab3187722b550baa45b2b0307b78132444b54c24eb36892c3d'), : RelatedNodeInfo(node_id='b70932b9-b9b2-4094-9e87-9cdc2ed6b90c', node_type=, metadata={}, hash='c8c0e5beea0643c8f86a849ae1c39c181029d182609af9b8dfc1a08d39c9b0b4')}, text='6\\nVector Institute | Annual Report 2022-2023 \\nONTARIO’S VIBRANT AI ECOSYSTEM\\nThe Vector Institute, in collaboration with Deloitte Canada, produces annual snapshots of the \\nOntario AI ecosystem, revealing its continued strength and role in nurturing top AI talent.\\nHIGHLIGHTS OF ONTARIO’S AI ECOSYSTEM\\nMore Ontario data will be available soon on the Vector Institute website . \\n20,634 \\nAI jobs created in Ontario75,975 \\nAI jobs retained in Ontario*1,503 \\nnew AI master’s and study \\npath enrollments1,050\\nnew AI master’s graduates from \\nVector-recognized programs248\\nnew AI-related patents filed\\nacross Canada\\n$1.16 BILLION\\nin AI-related Ontario \\nVC investments312\\ncompanies investe\\nd in the \\nOntario AI ecosystem23% \\nincrease in AI R&D spending or budgets*27\\nnew AI companies established in Ontario68%\\nof CEOs surveyed reported adopting AI solutions*\\n*In this report, Deloitte refined the methodology to improve accuracy of this calculation as well as validated it against other external data sources.', start_char_idx=0, end_char_idx=1009, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.9154188120411539),\n",
376 | " NodeWithScore(node=TextNode(id_='6af722ec-7d4a-40a9-80a0-d0c7919c5fd1', embedding=None, metadata={'page_label': '24', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={: RelatedNodeInfo(node_id='a51fcb51-0caa-44a8-ac08-46c0f9781b28', node_type=, metadata={'page_label': '24', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='2be0f87269fa3f0366e32912eb84d70c2e99facbfb9b15a32a59786d3f04c4dc'), : RelatedNodeInfo(node_id='d10dbc11-c5e3-445e-aa74-73bb7c44f7e5', node_type=, metadata={'page_label': '23', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='efbd640facc631c0c4ff11a69c807b2edf1204b53f3bcb9ca19afad9c7d5abab'), : RelatedNodeInfo(node_id='d21e5d50-1020-4491-987e-fec51f1c2f60', node_type=, metadata={}, hash='1978c85f73b84bc6fdce8b89f5475b65392371816a70973d65b3a6f5fd57e0c1')}, text='24\\nVector Institute | Annual Report 2022-2023 \\nTALENT AND WORKFORCE DEVELOPMENT\\nVECTOR’S CRUCIAL ROLE IN ADVANCING \\nONTARIO’S ECONOMY\\nVector works with both universities and employers to address the \\nincreasing demand for AI expertise by creating channels for promising students to access degree programs that cultivate the most sought-after AI skills in industry. \\nThrough Vector’s initiatives and experiential learning opportunities, \\nstudents learn crucial AI technical and job-related competencies, make connections with leading employers, and contribute to the constructive hiring outcomes that are reshaping Ontario’s economy. \\nAs of March 31, 2023, more than 1,325 students from \\nVector-recognized programs and AI study paths, including Vector Institute Scholarship in Artificial In elligence (VSAI) recipients, have been hired by Ontario employers.\"Ontario is a strong player in the field of artificial \\nintelligence, technology, and research, and we are thrilled to see so many new graduates looking to invest their skills and ideas in the province.\\n\"\\n-Vic Fedeli, Minister of Economic Development, Job Creationand Trade of Ontario*\\n*Quote source', start_char_idx=0, end_char_idx=1164, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.9083415588705721)]"
377 | ]
378 | },
379 | "execution_count": 14,
380 | "metadata": {},
381 | "output_type": "execute_result"
382 | }
383 | ],
384 | "source": [
385 | "# to view the retrieved nodes\n",
386 | "retrieved_nodes"
387 | ]
388 | },
389 | {
390 | "cell_type": "markdown",
391 | "id": "978ae2c5-8c2a-41c7-a2eb-85a5562f2db5",
392 | "metadata": {},
393 | "source": [
394 | "## 3. Generate Final Response\n",
395 | "\n",
396 | ""
397 | ]
398 | },
399 | {
400 | "cell_type": "code",
401 | "execution_count": 15,
402 | "id": "ef33c349-eed4-4e35-9b5d-9473adf2ce01",
403 | "metadata": {},
404 | "outputs": [],
405 | "source": [
406 | "\"\"\"Context-Augemented Generation.\n",
407 | "\n",
408 | "With our Index ready, we can create a QueryEngine\n",
409 | "that handles the retrieval and context augmentation\n",
410 | "in order to get the final response.\n",
411 | "\"\"\"\n",
412 | "\n",
413 | "query_engine = index.as_query_engine()"
414 | ]
415 | },
416 | {
417 | "cell_type": "code",
418 | "execution_count": 16,
419 | "id": "4139c48a-ece8-4244-b4eb-7cff74cb1325",
420 | "metadata": {},
421 | "outputs": [
422 | {
423 | "name": "stdout",
424 | "output_type": "stream",
425 | "text": [
426 | "Context information is below.\n",
427 | "---------------------\n",
428 | "{context_str}\n",
429 | "---------------------\n",
430 | "Given the context information and not prior knowledge, answer the query.\n",
431 | "Query: {query_str}\n",
432 | "Answer: \n"
433 | ]
434 | }
435 | ],
436 | "source": [
437 | "# to inspect the default prompt being used\n",
438 | "print(query_engine.\n",
439 | " get_prompts()[\"response_synthesizer:text_qa_template\"]\n",
440 | " .default_template.template\n",
441 | ")"
442 | ]
443 | },
444 | {
445 | "cell_type": "code",
446 | "execution_count": 17,
447 | "id": "6179639d-af96-4a09-b440-b47ad599a26f",
448 | "metadata": {},
449 | "outputs": [
450 | {
451 | "name": "stdout",
452 | "output_type": "stream",
453 | "text": [
454 | "20,634 AI jobs were created in Ontario according to Vector Institute's Annual Report 2022-2023.\n"
455 | ]
456 | }
457 | ],
458 | "source": [
459 | "response = query_engine.query(\n",
460 | " \"According to Vector Institute's Annual Report 2022-2023, \"\n",
461 | " \"how many AI jobs were created in Ontario?\"\n",
462 | ")\n",
463 | "print(response)"
464 | ]
465 | },
466 | {
467 | "cell_type": "markdown",
468 | "id": "a050bb7f-266c-46c7-8773-eccb909e5fd6",
469 | "metadata": {},
470 | "source": [
471 | "## More Queries"
472 | ]
473 | },
474 | {
475 | "cell_type": "markdown",
476 | "id": "20be9033-f240-4505-9761-29e9455c8974",
477 | "metadata": {},
478 | "source": [
479 | "### Comparisons"
480 | ]
481 | },
482 | {
483 | "cell_type": "code",
484 | "execution_count": 18,
485 | "id": "c86e99c2-601e-443e-aed2-ebe25106a966",
486 | "metadata": {},
487 | "outputs": [
488 | {
489 | "name": "stdout",
490 | "output_type": "stream",
491 | "text": [
492 | "27\n"
493 | ]
494 | }
495 | ],
496 | "source": [
497 | "query = (\n",
498 | " \"According to Vector Institute's 2022-2023 annual report, \"\n",
499 | " \"how many new AI companies were established in Ontario?\"\n",
500 | ")\n",
501 | "\n",
502 | "response = query_engine.query(query)\n",
503 | "print(response)"
504 | ]
505 | },
506 | {
507 | "cell_type": "code",
508 | "execution_count": 19,
509 | "id": "b02d9ae9-42ed-4f6e-8c2f-709d7e97dd22",
510 | "metadata": {},
511 | "outputs": [
512 | {
513 | "name": "stdout",
514 | "output_type": "stream",
515 | "text": [
516 | "In the years 2022 and 2023, the dollar value for Unrestricted net assets was $51,630,471 and $52,937,983, respectively.\n"
517 | ]
518 | }
519 | ],
520 | "source": [
521 | "query = (\n",
522 | " \"According to Vector Institute's 2022-2023 annual report, \"\n",
523 | " \"what was the dollar value for Unrestricted net assets in \"\n",
524 | " \"the years 2022 & 2023?\"\n",
525 | ")\n",
526 | "\n",
527 | "response = query_engine.query(query)\n",
528 | "print(response)"
529 | ]
530 | },
531 | {
532 | "cell_type": "code",
533 | "execution_count": 20,
534 | "id": "5bafbb1e-96e7-4a9f-909f-e7bfffd76532",
535 | "metadata": {},
536 | "outputs": [
537 | {
538 | "name": "stdout",
539 | "output_type": "stream",
540 | "text": [
541 | "According to Vector Institute's 2022-2023 annual report, the platinum sponsors were BMO Financial Group, Google, Loblaw Companies Ltd., NVIDIA, RBC, Scotiabank, Shopify Inc., and TD Bank Group.\n"
542 | ]
543 | }
544 | ],
545 | "source": [
546 | "query = (\n",
547 | " \"According to Vector Institute's 2022-2023 annual report, \"\n",
548 | " \"what companies were platinum sponsors?\"\n",
549 | ")\n",
550 | "\n",
551 | "response = query_engine.query(query)\n",
552 | "print(response)"
553 | ]
554 | },
555 | {
556 | "cell_type": "markdown",
557 | "id": "dae63946-be38-4807-af2a-8113661a806b",
558 | "metadata": {},
559 | "source": [
560 | "\n",
561 | "\n",
562 | "\n",
563 | "\n",
564 | "- LLMs as powerful as they are, don't perform too well with knowledge-intensive tasks (domain specific, updated data, long-tail)\n",
565 | "- Context augmentation has been shown (in a few studies) to outperform LLMs without augmentation\n",
566 | "- In this notebook, we showed one such example that follows that pattern.\n",
567 | "\n",
568 | ""
569 | ]
570 | },
571 | {
572 | "cell_type": "markdown",
573 | "id": "e4e03d54-2e9e-4ee4-bbee-8c5263e22014",
574 | "metadata": {},
575 | "source": [
576 | "\n",
577 | "\n",
578 | ""
579 | ]
580 | },
581 | {
582 | "cell_type": "markdown",
583 | "id": "0437c14b-5c7a-46c0-8315-19abe5a53430",
584 | "metadata": {},
585 | "source": [
586 | "\n",
587 | "\n",
588 | ""
589 | ]
590 | },
591 | {
592 | "cell_type": "markdown",
593 | "id": "d9790a7a-5e81-4383-9fe3-b48ac16d2044",
594 | "metadata": {},
595 | "source": [
596 | "\n",
597 | "\n",
598 | ""
599 | ]
600 | }
601 | ],
602 | "metadata": {
603 | "kernelspec": {
604 | "display_name": "mlops",
605 | "language": "python",
606 | "name": "mlops"
607 | },
608 | "language_info": {
609 | "codemirror_mode": {
610 | "name": "ipython",
611 | "version": 3
612 | },
613 | "file_extension": ".py",
614 | "mimetype": "text/x-python",
615 | "name": "python",
616 | "nbconvert_exporter": "python",
617 | "pygments_lexer": "ipython3",
618 | "version": "3.11.3"
619 | }
620 | },
621 | "nbformat": 4,
622 | "nbformat_minor": 5
623 | }
624 |
--------------------------------------------------------------------------------
/2024/otpp/otpp.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "5ab60f84-39b3-4bdd-ae83-6527acb315e5",
6 | "metadata": {},
7 | "source": [
8 | "# LLMs and LlamaIndex ◦ April 2 2024 ◦ Ontario Teacher's Pension Plan"
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": 1,
14 | "id": "2276edc2-74ce-4d85-bdaa-0607c74fc981",
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "##################################################################\n",
19 | "# Venue: OTPP L&L\n",
20 | "# Talk: LLMs and LlamaIndex\n",
21 | "# Speaker: Andrei Fajardo\n",
22 | "##################################################################"
23 | ]
24 | },
25 | {
26 | "cell_type": "markdown",
27 | "id": "3bee89b8-a04d-4326-9392-b9e7e1bcb8af",
28 | "metadata": {},
29 | "source": [
30 | ""
31 | ]
32 | },
33 | {
34 | "cell_type": "markdown",
35 | "id": "e4d38b38-ea48-4012-81ae-84e1d1f40a69",
36 | "metadata": {},
37 | "source": [
38 | ""
39 | ]
40 | },
41 | {
42 | "cell_type": "markdown",
43 | "id": "34d1f8e7-f978-4f19-bdfb-37c0d235b5bf",
44 | "metadata": {},
45 | "source": [
46 | "#### Notebook Setup & Dependency Installation"
47 | ]
48 | },
49 | {
50 | "cell_type": "code",
51 | "execution_count": 2,
52 | "id": "f227a52a-a147-4e8f-b7d3-e03f983fd5f1",
53 | "metadata": {
54 | "scrolled": true
55 | },
56 | "outputs": [
57 | {
58 | "name": "stdout",
59 | "output_type": "stream",
60 | "text": [
61 | "\n",
62 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
63 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.11 -m pip install --upgrade pip\u001b[0m\n",
64 | "Note: you may need to restart the kernel to use updated packages.\n"
65 | ]
66 | }
67 | ],
68 | "source": [
69 | "%pip install llama-index-vector-stores-qdrant -q"
70 | ]
71 | },
72 | {
73 | "cell_type": "code",
74 | "execution_count": 4,
75 | "id": "7bc383fc-19b2-47b5-af61-e83210ea9c37",
76 | "metadata": {},
77 | "outputs": [],
78 | "source": [
79 | "import nest_asyncio\n",
80 | "\n",
81 | "nest_asyncio.apply()"
82 | ]
83 | },
84 | {
85 | "cell_type": "code",
86 | "execution_count": 31,
87 | "id": "d56aba9c-230b-464d-909c-de17d1bc0aa6",
88 | "metadata": {},
89 | "outputs": [
90 | {
91 | "name": "stdout",
92 | "output_type": "stream",
93 | "text": [
94 | "mkdir: data: File exists\n",
95 | "--2024-04-03 11:23:36-- https://www.otpp.com/content/dam/otpp/documents/reports/2023-ar/otpp-2023-annual-report-eng.pdf\n",
96 | "Resolving www.otpp.com (www.otpp.com)... 67.210.219.20\n",
97 | "Connecting to www.otpp.com (www.otpp.com)|67.210.219.20|:443... connected.\n",
98 | "HTTP request sent, awaiting response... 200 OK\n",
99 | "Length: 10901939 (10M) [application/pdf]\n",
100 | "Saving to: ‘./data/otpp-2023-annual-report-eng.pdf’\n",
101 | "\n",
102 | "./data/otpp-2023-an 100%[===================>] 10.40M 1.01MB/s in 10s \n",
103 | "\n",
104 | "2024-04-03 11:23:49 (1.02 MB/s) - ‘./data/otpp-2023-annual-report-eng.pdf’ saved [10901939/10901939]\n",
105 | "\n"
106 | ]
107 | }
108 | ],
109 | "source": [
110 | "!mkdir data\n",
111 | "!wget \"https://www.otpp.com/content/dam/otpp/documents/reports/2023-ar/otpp-2023-annual-report-eng.pdf\" -O \"./data/otpp-2023-annual-report-eng.pdf\""
112 | ]
113 | },
114 | {
115 | "cell_type": "markdown",
116 | "id": "275c00f1-e358-498a-88c3-8e810a5a2546",
117 | "metadata": {},
118 | "source": [
119 | "## Motivation\n",
120 | "\n",
121 | "\n"
122 | ]
123 | },
124 | {
125 | "cell_type": "code",
126 | "execution_count": 5,
127 | "id": "25d4ce76-8eea-44cb-aa99-94844dfed9c7",
128 | "metadata": {},
129 | "outputs": [],
130 | "source": [
131 | "# query an LLM and ask it about Mistplay\n",
132 | "from llama_index.llms.openai import OpenAI\n",
133 | "\n",
134 | "llm = OpenAI(model=\"gpt-4-turbo-preview\")\n",
135 | "response = llm.complete(\"What is Ontario Teacher's Pension Plan all about?\")"
136 | ]
137 | },
138 | {
139 | "cell_type": "code",
140 | "execution_count": 6,
141 | "id": "c3f18489-4f25-40ce-86e9-697ddea7d6c6",
142 | "metadata": {},
143 | "outputs": [
144 | {
145 | "name": "stdout",
146 | "output_type": "stream",
147 | "text": [
148 | "The Ontario Teachers' Pension Plan (OTPP) is one of the world's largest pension funds, serving the public school teachers of Ontario, Canada. Established in 1990, it operates as an independent organization responsible for administering defined-benefit pensions for school teachers of the province. The OTPP is jointly sponsored by the Government of Ontario and the Ontario Teachers' Federation, meaning both the government and the teachers' union have a say in the management and direction of the fund.\n",
149 | "\n",
150 | "The primary purpose of the OTPP is to provide a stable and reliable source of retirement income for its members. It does so by collecting contributions from both teachers and their employers (the government) and investing those funds in a wide variety of assets, including stocks, bonds, real estate, and infrastructure projects, both domestically and internationally. The goal is to generate sufficient returns to ensure the long-term sustainability of the pension plan while managing risk.\n",
151 | "\n",
152 | "The OTPP is notable for its size, investment success, and innovative approach to pension management. It has been a pioneer in direct investment, taking significant stakes in companies, real estate, and infrastructure projects around the world. This direct investment strategy has allowed the OTPP to reduce reliance on external fund managers, thereby lowering costs and potentially increasing returns.\n",
153 | "\n",
154 | "The plan's governance model is designed to ensure stability and sustainability. The board of directors, which oversees the operation of the OTPP, includes representatives from both the government and the Ontario Teachers' Federation. This joint sponsorship model ensures that both the interests of the teachers and the fiscal responsibilities of the government are taken into account in the plan's management.\n",
155 | "\n",
156 | "In summary, the Ontario Teachers' Pension Plan is a critical component of the retirement security system for Ontario's public school teachers, distinguished by its large scale, sophisticated investment strategy, and joint governance structure. It aims to deliver secure and sustainable pensions to its members, contributing significantly to their financial well-being in retirement.\n"
157 | ]
158 | }
159 | ],
160 | "source": [
161 | "print(response)"
162 | ]
163 | },
164 | {
165 | "cell_type": "code",
166 | "execution_count": 7,
167 | "id": "80b2f9c3-1e07-4178-9e7e-220355725955",
168 | "metadata": {},
169 | "outputs": [],
170 | "source": [
171 | "response = llm.complete(\"According to the 2023 annual report, how many billions of dollars in net assets does Ontario Teacher's Pension Plan hold?\")"
172 | ]
173 | },
174 | {
175 | "cell_type": "code",
176 | "execution_count": 8,
177 | "id": "6a1c6424-cf2f-4ee1-b601-674b03354e35",
178 | "metadata": {},
179 | "outputs": [
180 | {
181 | "name": "stdout",
182 | "output_type": "stream",
183 | "text": [
184 | "As of my last update in April 2023, the Ontario Teachers' Pension Plan (OTPP) reported holding net assets of CAD 247.5 billion in its 2022 annual report. Please note that this information might have changed after my last update, and I would recommend checking the latest annual report or official OTPP sources for the most current figures.\n"
185 | ]
186 | }
187 | ],
188 | "source": [
189 | "print(response)"
190 | ]
191 | },
192 | {
193 | "cell_type": "code",
194 | "execution_count": 9,
195 | "id": "632e1dec-045b-42fb-b325-c25e1ec4bb75",
196 | "metadata": {},
197 | "outputs": [],
198 | "source": [
199 | "response = llm.complete(\"According to the 2023 annual report, what is the 10-year total-fund net return?\")"
200 | ]
201 | },
202 | {
203 | "cell_type": "code",
204 | "execution_count": 10,
205 | "id": "53efac56-7966-407f-a298-bc2e0e14518e",
206 | "metadata": {},
207 | "outputs": [
208 | {
209 | "name": "stdout",
210 | "output_type": "stream",
211 | "text": [
212 | "As of my last update in April 2023, I don't have access to real-time data or specific annual reports from 2023. Therefore, I cannot provide the 10-year total-fund net return from any specific 2023 annual report. This information would typically be found directly in the report of the specific fund or investment you're interested in. I recommend checking the official website or contacting the financial institution directly for the most accurate and up-to-date information.\n"
213 | ]
214 | }
215 | ],
216 | "source": [
217 | "print(response)"
218 | ]
219 | },
220 | {
221 | "cell_type": "markdown",
222 | "id": "04a0ef8d-d55c-4b64-887b-18d343503a76",
223 | "metadata": {},
224 | "source": [
225 | "## Basic RAG in 3 Steps\n",
226 | "\n",
227 | "\n",
228 | "\n",
229 | "\n",
230 | "1. Build external knowledge (i.e., uploading updated data sources)\n",
231 | "2. Retrieve\n",
232 | "3. Augment and Generate"
233 | ]
234 | },
235 | {
236 | "cell_type": "markdown",
237 | "id": "598a5257-20ae-468e-85d6-d4e8c46b8cb5",
238 | "metadata": {},
239 | "source": [
240 | "## 1. Build External Knowledge\n",
241 | "\n",
242 | ""
243 | ]
244 | },
245 | {
246 | "cell_type": "code",
247 | "execution_count": 11,
248 | "id": "a2963f90-9da5-4a0d-8dbe-f16fcb8627a3",
249 | "metadata": {},
250 | "outputs": [],
251 | "source": [
252 | "\"\"\"Load the data.\n",
253 | "\n",
254 | "With llama-index, before any transformations are applied,\n",
255 | "data is loaded in the `Document` abstraction, which is\n",
256 | "a container that holds the text of the document.\n",
257 | "\"\"\"\n",
258 | "\n",
259 | "from llama_index.core import SimpleDirectoryReader\n",
260 | "\n",
261 | "loader = SimpleDirectoryReader(input_dir=\"./data\")\n",
262 | "documents = loader.load_data()"
263 | ]
264 | },
265 | {
266 | "cell_type": "code",
267 | "execution_count": 12,
268 | "id": "da321e2c-8428-4c04-abf2-b204416e816f",
269 | "metadata": {},
270 | "outputs": [
271 | {
272 | "data": {
273 | "text/plain": [
274 | "'Investing to \\nmake a mark\\n2023 Annual Report'"
275 | ]
276 | },
277 | "execution_count": 12,
278 | "metadata": {},
279 | "output_type": "execute_result"
280 | }
281 | ],
282 | "source": [
283 | "# if you want to see what the text looks like\n",
284 | "documents[0].text[:1000]"
285 | ]
286 | },
287 | {
288 | "cell_type": "code",
289 | "execution_count": 13,
290 | "id": "4801e74a-8c52-45c4-967d-7a1a94f54ad3",
291 | "metadata": {},
292 | "outputs": [],
293 | "source": [
294 | "\"\"\"Chunk, Encode, and Store into a Vector Store.\n",
295 | "\n",
296 | "To streamline the process, we can make use of the IngestionPipeline\n",
297 | "class that will apply your specified transformations to the\n",
298 | "Document's.\n",
299 | "\"\"\"\n",
300 | "\n",
301 | "from llama_index.core.ingestion import IngestionPipeline\n",
302 | "from llama_index.core.node_parser import SentenceSplitter\n",
303 | "from llama_index.embeddings.openai import OpenAIEmbedding\n",
304 | "from llama_index.vector_stores.qdrant import QdrantVectorStore\n",
305 | "import qdrant_client\n",
306 | "\n",
307 | "client = qdrant_client.QdrantClient(location=\":memory:\")\n",
308 | "vector_store = QdrantVectorStore(client=client, collection_name=\"test_store\")\n",
309 | "\n",
310 | "pipeline = IngestionPipeline(\n",
311 | " transformations = [\n",
312 | " SentenceSplitter(),\n",
313 | " OpenAIEmbedding(),\n",
314 | " ],\n",
315 | " vector_store=vector_store,\n",
316 | ")\n",
317 | "_nodes = pipeline.run(documents=documents, num_workers=4)"
318 | ]
319 | },
320 | {
321 | "cell_type": "code",
322 | "execution_count": 14,
323 | "id": "02afea25-098b-49c7-a965-21c7576757af",
324 | "metadata": {},
325 | "outputs": [
326 | {
327 | "data": {
328 | "text/plain": [
329 | "'Investing to \\nmake a mark\\n2023 Annual Report'"
330 | ]
331 | },
332 | "execution_count": 14,
333 | "metadata": {},
334 | "output_type": "execute_result"
335 | }
336 | ],
337 | "source": [
338 | "# if you want to see the nodes\n",
339 | "# len(_nodes)\n",
340 | "_nodes[0].text"
341 | ]
342 | },
343 | {
344 | "cell_type": "code",
345 | "execution_count": 15,
346 | "id": "44cd8a86-089d-4329-9484-35b98b3a26f9",
347 | "metadata": {},
348 | "outputs": [],
349 | "source": [
350 | "\"\"\"Create a llama-index... wait for it... Index.\n",
351 | "\n",
352 | "After uploading your encoded documents into your vector\n",
353 | "store of choice, you can connect to it with a VectorStoreIndex\n",
354 | "which then gives you access to all of the llama-index functionality.\n",
355 | "\"\"\"\n",
356 | "\n",
357 | "from llama_index.core import VectorStoreIndex\n",
358 | "\n",
359 | "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)"
360 | ]
361 | },
362 | {
363 | "cell_type": "markdown",
364 | "id": "286b1827-7547-49c6-aba3-82f08d6d86b8",
365 | "metadata": {},
366 | "source": [
367 | "## 2. Retrieve Against A Query\n",
368 | "\n",
369 | ""
370 | ]
371 | },
372 | {
373 | "cell_type": "code",
374 | "execution_count": 16,
375 | "id": "49f86af1-db08-4641-89ad-d60abd04e6b3",
376 | "metadata": {},
377 | "outputs": [],
378 | "source": [
379 | "\"\"\"Retrieve relevant documents against a query.\n",
380 | "\n",
381 | "With our Index ready, we can now query it to\n",
382 | "retrieve the most relevant document chunks.\n",
383 | "\"\"\"\n",
384 | "\n",
385 | "retriever = index.as_retriever(similarity_top_k=2)\n",
386 | "retrieved_nodes = retriever.retrieve(\"According to the 2023 annual report, what is the 10-year total-fund net return?\")"
387 | ]
388 | },
389 | {
390 | "cell_type": "code",
391 | "execution_count": 17,
392 | "id": "05f9ce3b-a4e3-4862-b58c-2d9fba1f9abc",
393 | "metadata": {},
394 | "outputs": [
395 | {
396 | "name": "stdout",
397 | "output_type": "stream",
398 | "text": [
399 | "Stable long-term total-fund returns \n",
400 | "1 Net assets include investment assets less investment liabilities (net investments), plus the receivables from the Province of Ontario, and \n",
401 | "other assets less other liabilities.\n",
402 | "2 A real rate of return is the net return, or annual percentage of profit earned on an investment, adjusted for inflation.Ontario Teachers’ investment program is tailored to \n",
403 | "generate strong and steady risk-adjusted returns to \n",
404 | "pay members’ pensions over generations, while also \n",
405 | "havi\n"
406 | ]
407 | }
408 | ],
409 | "source": [
410 | "# to view the retrieved node\n",
411 | "print(retrieved_nodes[0].text[:500])"
412 | ]
413 | },
414 | {
415 | "cell_type": "markdown",
416 | "id": "978ae2c5-8c2a-41c7-a2eb-85a5562f2db5",
417 | "metadata": {},
418 | "source": [
419 | "## 3. Generate Final Response\n",
420 | "\n",
421 | ""
422 | ]
423 | },
424 | {
425 | "cell_type": "code",
426 | "execution_count": 18,
427 | "id": "ef33c349-eed4-4e35-9b5d-9473adf2ce01",
428 | "metadata": {},
429 | "outputs": [],
430 | "source": [
431 | "\"\"\"Context-Augemented Generation.\n",
432 | "\n",
433 | "With our Index ready, we can create a QueryEngine\n",
434 | "that handles the retrieval and context augmentation\n",
435 | "in order to get the final response.\n",
436 | "\"\"\"\n",
437 | "\n",
438 | "query_engine = index.as_query_engine()"
439 | ]
440 | },
441 | {
442 | "cell_type": "code",
443 | "execution_count": 19,
444 | "id": "4139c48a-ece8-4244-b4eb-7cff74cb1325",
445 | "metadata": {},
446 | "outputs": [
447 | {
448 | "name": "stdout",
449 | "output_type": "stream",
450 | "text": [
451 | "Context information is below.\n",
452 | "---------------------\n",
453 | "{context_str}\n",
454 | "---------------------\n",
455 | "Given the context information and not prior knowledge, answer the query.\n",
456 | "Query: {query_str}\n",
457 | "Answer: \n"
458 | ]
459 | }
460 | ],
461 | "source": [
462 | "# to inspect the default prompt being used\n",
463 | "print(query_engine.\n",
464 | " get_prompts()[\"response_synthesizer:text_qa_template\"]\n",
465 | " .default_template.template\n",
466 | ")"
467 | ]
468 | },
469 | {
470 | "cell_type": "code",
471 | "execution_count": 20,
472 | "id": "6179639d-af96-4a09-b440-b47ad599a26f",
473 | "metadata": {},
474 | "outputs": [
475 | {
476 | "name": "stdout",
477 | "output_type": "stream",
478 | "text": [
479 | "The 10-year total-fund net return, as stated in the 2023 annual report, is 7.6%.\n"
480 | ]
481 | }
482 | ],
483 | "source": [
484 | "response = query_engine.query(\"According to the 2023 annual report, what is the 10-year total-fund net return?\")\n",
485 | "print(response)"
486 | ]
487 | },
488 | {
489 | "cell_type": "markdown",
490 | "id": "a050bb7f-266c-46c7-8773-eccb909e5fd6",
491 | "metadata": {},
492 | "source": [
493 | "## Beyond Basic RAG: Improved PDF Parsing with LlamaParse\n",
494 | "\n",
495 | "To use LlamaParse, you first need to obtain an API Key. Visit [llamacloud.ai](https://cloud.llamaindex.ai/login) to login (or sign-up) and get an api key."
496 | ]
497 | },
498 | {
499 | "cell_type": "code",
500 | "execution_count": null,
501 | "id": "281b527d-a305-4432-b34f-811860ed6166",
502 | "metadata": {},
503 | "outputs": [],
504 | "source": [
505 | "api_key="
506 | ]
507 | },
508 | {
509 | "cell_type": "markdown",
510 | "id": "a4a08bc5-1ff9-42f5-91a3-f77f3a529ff1",
511 | "metadata": {},
512 | "source": [
513 | "### The default pdf reader (PyPDF), like many out-of-the box pdf parsers struggle on complex PDF docs."
514 | ]
515 | },
516 | {
517 | "cell_type": "code",
518 | "execution_count": 21,
519 | "id": "32b6d6bd-f04a-400a-99ac-b9c0308c7554",
520 | "metadata": {},
521 | "outputs": [
522 | {
523 | "name": "stdout",
524 | "output_type": "stream",
525 | "text": [
526 | "Steve McGirr, Chair of the Board, attended all board meetings, which totaled 11 full meetings, including one strategic offsite meeting.\n"
527 | ]
528 | }
529 | ],
530 | "source": [
531 | "response = query_engine.query(\"How many board meetings did Steve McGirr, Chair of the Board, attend?\")\n",
532 | "print(response)"
533 | ]
534 | },
535 | {
536 | "cell_type": "code",
537 | "execution_count": 24,
538 | "id": "db781501-1e4f-4c2f-8ba5-aef3ab9c693e",
539 | "metadata": {},
540 | "outputs": [
541 | {
542 | "name": "stdout",
543 | "output_type": "stream",
544 | "text": [
545 | "55% of board members identify as women.\n"
546 | ]
547 | }
548 | ],
549 | "source": [
550 | "response = query_engine.query(\"What percentage of board members identify as women?\")\n",
551 | "print(response)"
552 | ]
553 | },
554 | {
555 | "cell_type": "code",
556 | "execution_count": 25,
557 | "id": "40e70776-611d-4b52-bd89-3f233cdaa9c5",
558 | "metadata": {},
559 | "outputs": [
560 | {
561 | "name": "stdout",
562 | "output_type": "stream",
563 | "text": [
564 | "The total investment percentage in Canada as of December 31, 2023, is 29% (10% in public equity and 19% in inflation-sensitive investments).\n"
565 | ]
566 | }
567 | ],
568 | "source": [
569 | "response = query_engine.query(\"What is the total investment percentage in Canada as of December 31, 2023?\")\n",
570 | "print(response)"
571 | ]
572 | },
573 | {
574 | "cell_type": "markdown",
575 | "id": "3895970c-dae2-4ae3-b48d-72da7617fcd3",
576 | "metadata": {},
577 | "source": [
578 | "### Improved PDF Parsing using LlamaParse"
579 | ]
580 | },
581 | {
582 | "cell_type": "code",
583 | "execution_count": 26,
584 | "id": "27556916-9e45-4bfe-b1d9-ea12ed4bab41",
585 | "metadata": {},
586 | "outputs": [],
587 | "source": [
588 | "from llama_parse import LlamaParse\n",
589 | "from llama_index.core.node_parser import MarkdownElementNodeParser\n",
590 | "from llama_index.llms.openai import OpenAI"
591 | ]
592 | },
593 | {
594 | "cell_type": "code",
595 | "execution_count": 29,
596 | "id": "ad81e2cf-90b6-49af-b36e-2f896abec0c8",
597 | "metadata": {},
598 | "outputs": [
599 | {
600 | "name": "stdout",
601 | "output_type": "stream",
602 | "text": [
603 | "Started parsing the file under job_id ddcdc5f9-bd16-40b8-90f2-353f2a2b6450\n"
604 | ]
605 | }
606 | ],
607 | "source": [
608 | "from llama_parse import LlamaParse\n",
609 | "\n",
610 | "parser = LlamaParse(result_type=\"markdown\")\n",
611 | "md_documents = parser.load_data(file_path='./data/otpp-2023-annual-report-eng.pdf')"
612 | ]
613 | },
614 | {
615 | "cell_type": "code",
616 | "execution_count": 30,
617 | "id": "c49c7404-1ff9-4cdc-bcc1-5a43378f21ac",
618 | "metadata": {},
619 | "outputs": [],
620 | "source": [
621 | "# save to an .md file\n",
622 | "with open(\"./mds/parsed.md\", \"w\") as f:\n",
623 | " f.write(md_documents[0].text)"
624 | ]
625 | },
626 | {
627 | "cell_type": "code",
628 | "execution_count": 29,
629 | "id": "3c84a614-b916-463b-bf58-ba2a4bced4b3",
630 | "metadata": {},
631 | "outputs": [
632 | {
633 | "name": "stderr",
634 | "output_type": "stream",
635 | "text": [
636 | "133it [00:00, 70142.39it/s]\n",
637 | "100%|███████████████████████████████████████████████████████████████████████████████████| 133/133 [01:13<00:00, 1.80it/s]\n"
638 | ]
639 | }
640 | ],
641 | "source": [
642 | "md_node_parser = MarkdownElementNodeParser(llm=OpenAI(model=\"gpt-4.5-turbo-preview\"), num_workers=3, include_metadata=True)\n",
643 | "md_nodes = md_node_parser.get_nodes_from_documents(md_documents)"
644 | ]
645 | },
646 | {
647 | "cell_type": "code",
648 | "execution_count": 34,
649 | "id": "efaed5ae-9c87-4670-b571-9dff91b06cea",
650 | "metadata": {},
651 | "outputs": [],
652 | "source": [
653 | "llama_parse_index = VectorStoreIndex.from_documents(md_documents)\n",
654 | "llama_parse_query_engine = llama_parse_index.as_query_engine()"
655 | ]
656 | },
657 | {
658 | "cell_type": "code",
659 | "execution_count": 43,
660 | "id": "3e54ff4a-22d8-45b4-b27c-84387ff7b4bb",
661 | "metadata": {},
662 | "outputs": [
663 | {
664 | "name": "stdout",
665 | "output_type": "stream",
666 | "text": [
667 | "Steve McGirr, Chair of the Board, attended 11 board meetings.\n"
668 | ]
669 | }
670 | ],
671 | "source": [
672 | "response = llama_parse_query_engine.query(\"How many board meetings did Steve McGirr, Chair of the Board, attend?\")\n",
673 | "print(response)"
674 | ]
675 | },
676 | {
677 | "cell_type": "code",
678 | "execution_count": 52,
679 | "id": "bdf74b49-a64e-4de2-a761-e117123ff069",
680 | "metadata": {},
681 | "outputs": [
682 | {
683 | "name": "stdout",
684 | "output_type": "stream",
685 | "text": [
686 | "55% of board members identify as women.\n"
687 | ]
688 | }
689 | ],
690 | "source": [
691 | "response = llama_parse_query_engine.query(\"What percentage of board members identify as women?\")\n",
692 | "print(response)"
693 | ]
694 | },
695 | {
696 | "cell_type": "code",
697 | "execution_count": 36,
698 | "id": "328ee422-cd6d-4d5d-bfe9-da5e878f85a5",
699 | "metadata": {},
700 | "outputs": [
701 | {
702 | "name": "stdout",
703 | "output_type": "stream",
704 | "text": [
705 | "The total investment percentage in Canada as of December 31, 2023, is 35%.\n"
706 | ]
707 | }
708 | ],
709 | "source": [
710 | "response = llama_parse_query_engine.query(\"What is the total investment percentage in Canada as of December 31, 2023?\")\n",
711 | "print(response)"
712 | ]
713 | },
714 | {
715 | "cell_type": "markdown",
716 | "id": "dae63946-be38-4807-af2a-8113661a806b",
717 | "metadata": {},
718 | "source": [
719 | "## In Summary\n",
720 | "\n",
721 | "- LLMs as powerful as they are, don't perform too well with knowledge-intensive tasks (domain specific, updated data, long-tail)\n",
722 | "- Context augmentation has been shown (in a few studies) to outperform LLMs without augmentation\n",
723 | "- In this notebook, we showed one such example that follows that pattern."
724 | ]
725 | },
726 | {
727 | "cell_type": "markdown",
728 | "id": "53bfd859-924a-4748-b4d8-ee0f2dc52e3b",
729 | "metadata": {},
730 | "source": [
731 | "## Data Extraction"
732 | ]
733 | },
734 | {
735 | "cell_type": "markdown",
736 | "id": "01fb77fd-f4e8-42b0-9704-d2229f4e66a8",
737 | "metadata": {},
738 | "source": [
739 | ""
740 | ]
741 | },
742 | {
743 | "cell_type": "code",
744 | "execution_count": 63,
745 | "id": "a63649cf-7f3a-4f6d-a383-92163f71b03c",
746 | "metadata": {},
747 | "outputs": [],
748 | "source": [
749 | "import json\n",
750 | "from llama_index.core.bridge.pydantic import BaseModel, Field\n",
751 | "from typing import List\n",
752 | "\n",
753 | "from llama_index.program.openai import OpenAIPydanticProgram\n",
754 | "from llama_index.llms.openai import OpenAI"
755 | ]
756 | },
757 | {
758 | "cell_type": "markdown",
759 | "id": "7f306da0-4da6-45b5-b9b3-50d8d0b9883a",
760 | "metadata": {},
761 | "source": [
762 | "### Leadership Team"
763 | ]
764 | },
765 | {
766 | "cell_type": "code",
767 | "execution_count": 70,
768 | "id": "b37bc7e7-4ee8-40e8-891f-0307997049a7",
769 | "metadata": {},
770 | "outputs": [],
771 | "source": [
772 | "class LeadershipTeam(BaseModel):\n",
773 | " \"\"\"Data model for leadership team.\"\"\"\n",
774 | " \n",
775 | " ceo: str = Field(description=\"The CEO\")\n",
776 | " coo: str = Field(description=\"The Chief Operating Officer\")\n",
777 | " cio: str = Field(description=\"Chief Investment Officer\")\n",
778 | " chief_pension_officer: str = Field(description=\"Chief Pension Officer\")\n",
779 | " chief_legal_officer: str = Field(description=\"Chief Legal & Corporate Affairs Officer\")\n",
780 | " chief_people_officer: str = Field(description=\"Chief People Officer\")\n",
781 | " chief_strategy_officer: str = Field(description=\"Chief Strategy Officer\")\n",
782 | " executive_managing_director: str = Field(description=\"Executive Managing Director\")\n",
783 | " chief_investment_officer: str = Field(description=\"Chief Investment Officer\")"
784 | ]
785 | },
786 | {
787 | "cell_type": "code",
788 | "execution_count": 71,
789 | "id": "ec28d600-5001-4176-bd9f-5982c05d02a3",
790 | "metadata": {},
791 | "outputs": [],
792 | "source": [
793 | "prompt_template_str = \"\"\"\\\n",
794 | "Here is the 2023 Annual Report for Ontario Teacher's Pension Plan:\n",
795 | "{document_text}\n",
796 | "Provide the names of the Leadership Team.\n",
797 | "\"\"\"\n",
798 | "\n",
799 | "program = OpenAIPydanticProgram.from_defaults(\n",
800 | " output_cls=LeadershipTeam,\n",
801 | " prompt_template_str=prompt_template_str,\n",
802 | " llm=OpenAI(\"gpt-4-turbo-preview\"),\n",
803 | " verbose=True\n",
804 | ")"
805 | ]
806 | },
807 | {
808 | "cell_type": "code",
809 | "execution_count": 72,
810 | "id": "0de616a2-c96a-4be0-b8ba-04e537d03e61",
811 | "metadata": {},
812 | "outputs": [
813 | {
814 | "name": "stdout",
815 | "output_type": "stream",
816 | "text": [
817 | "Function call: LeadershipTeam with args: {\"ceo\":\"Jo Taylor\",\"coo\":\"Tracy Abel\",\"cio\":\"Gillian Brown\",\"chief_pension_officer\":\"Charley Butler\",\"chief_legal_officer\":\"Sharon Chilcott\",\"chief_people_officer\":\"Jeff Davis\",\"chief_strategy_officer\":\"Jonathan Hausman\",\"executive_managing_director\":\"Nick Jansa\",\"chief_investment_officer\":\"Stephen McLennan\"}\n"
818 | ]
819 | }
820 | ],
821 | "source": [
822 | "leadership_team = program(document_text=md_documents[0].text)"
823 | ]
824 | },
825 | {
826 | "cell_type": "code",
827 | "execution_count": 73,
828 | "id": "41f8b4b0-0f19-4101-9642-1b5bc555dfac",
829 | "metadata": {},
830 | "outputs": [
831 | {
832 | "name": "stdout",
833 | "output_type": "stream",
834 | "text": [
835 | "{\n",
836 | " \"ceo\": \"Jo Taylor\",\n",
837 | " \"coo\": \"Tracy Abel\",\n",
838 | " \"cio\": \"Gillian Brown\",\n",
839 | " \"chief_pension_officer\": \"Charley Butler\",\n",
840 | " \"chief_legal_officer\": \"Sharon Chilcott\",\n",
841 | " \"chief_people_officer\": \"Jeff Davis\",\n",
842 | " \"chief_strategy_officer\": \"Jonathan Hausman\",\n",
843 | " \"executive_managing_director\": \"Nick Jansa\",\n",
844 | " \"chief_investment_officer\": \"Stephen McLennan\"\n",
845 | "}\n"
846 | ]
847 | }
848 | ],
849 | "source": [
850 | "print(json.dumps(leadership_team.dict(), indent=4))"
851 | ]
852 | },
853 | {
854 | "cell_type": "markdown",
855 | "id": "fc857227-3fed-4bb6-a062-99ea3c55e294",
856 | "metadata": {},
857 | "source": [
858 | "# LlamaIndex Has More To Offer\n",
859 | "\n",
860 | "- Data infrastructure that enables production-grade, advanced RAG systems\n",
861 | "- Agentic solutions\n",
862 | "- Newly released: `llama-index-networks`\n",
863 | "- Enterprise offerings (alpha):\n",
864 | " - LlamaParse (proprietary complex PDF parser) and\n",
865 | " - LlamaCloud"
866 | ]
867 | },
868 | {
869 | "cell_type": "markdown",
870 | "id": "17c1c027-be8b-48f4-87ee-06f3e2c71797",
871 | "metadata": {},
872 | "source": [
873 | "### Useful links\n",
874 | "\n",
875 | "[website](https://www.llamaindex.ai/) ◦ [llamahub](https://llamahub.ai) ◦ [llamaparse](https://github.com/run-llama/llama_parse) ◦ [github](https://github.com/run-llama/llama_index) ◦ [medium](https://medium.com/@llama_index) ◦ [rag-bootcamp-poster](https://d3ddy8balm3goa.cloudfront.net/rag-bootcamp-vector/final_poster.excalidraw.svg)"
876 | ]
877 | }
878 | ],
879 | "metadata": {
880 | "kernelspec": {
881 | "display_name": "otpp",
882 | "language": "python",
883 | "name": "otpp"
884 | },
885 | "language_info": {
886 | "codemirror_mode": {
887 | "name": "ipython",
888 | "version": 3
889 | },
890 | "file_extension": ".py",
891 | "mimetype": "text/x-python",
892 | "name": "python",
893 | "nbconvert_exporter": "python",
894 | "pygments_lexer": "ipython3",
895 | "version": "3.11.3"
896 | }
897 | },
898 | "nbformat": 4,
899 | "nbformat_minor": 5
900 | }
901 |
--------------------------------------------------------------------------------
/2024/rbc/TODO.txt:
--------------------------------------------------------------------------------
1 | Examples:
2 |
3 | 1. A Gang of LLMs Tell a Story (done! add money to anthropic account)
4 | 2. LLMs Lack Access To Updated Data (done!)
5 | 3. RAG Yields More Accurate Responses (done!)
6 | 4. 3 Steps For Building Basic RAG (done!)
7 | 5. Graph RAG (done!)
8 | 6. Agent Ingredients: Tool Use (done!)
9 | 7. Agent Ingredients: Memory (done!)
10 | 8. Agent Reflection: Toxicity Reduction (done!)
11 | 9. Agentic RAG (done!)
12 | 10. LlamaHub Discoverability: Readers, Packs more (done!)
--------------------------------------------------------------------------------
/2024/vector-oss-tools/kg.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
16 |
22 |
27 |
28 |
29 |
30 |
31 |
32 |
118 |
119 |
120 |
121 |
122 |
127 |
128 |
129 |
137 |
138 |
139 |
140 |
249 |
250 |
--------------------------------------------------------------------------------
/2024/vector-pe-lab/manga/manga.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/nerdai/talks/14c51331ffe8eb357f4d596114b5b4d1f8785a41/2024/vector-pe-lab/manga/manga.pdf
--------------------------------------------------------------------------------
/2024/vector-pe-lab/mds/instruction-parsed.md:
--------------------------------------------------------------------------------
1 | # Manga Comic Book
2 |
3 | COMics inside!
4 |
5 | HIROYUKI KOJIMA
6 |
7 | SHIN TOGAMI
8 |
9 | BecomCo, LTD:
10 |
11 | no starch
12 |
13 | press
14 |
15 | Ohmsha
16 | ---
17 | # The Asagake Times
18 |
19 | Sanda-Cho Distributor
20 |
21 | A newspaper distributor?
22 |
23 | do I have the wrong map?
24 |
25 | You're looking for the Sanda-cho branch office?
26 |
27 | It's next door.
28 |
29 | Everybody mistakes us for the office because we are larger.
30 |
31 | What Is a Function? 3
32 | ---
33 | # Derivative Calculation
34 |
35 | Let’s find the derivative of constant function f(x) = α. The differential coefficient of f(x) at x = a is
36 |
37 | lim f(a + ε) − f(a) = lim α − α = lim 0 = 0
38 |
39 | ε→0 ε ε→0 ε ε→0
40 |
41 | Thus, the derivative of f(x) is f′(x) = 0. This makes sense, since our function is constant—the rate of change is 0.
42 |
43 | Note: The differential coefficient of f(x) at x = a is often simply called the derivative of f(x) at x = a, or just f′(a).
44 |
45 | Let’s calculate the derivative of linear function f(x) = αx + β. The derivative of f(x) at x = α is
46 |
47 | lim f(α + ε) − f(a) = lim α(a + ε) + β − (αa + β) = lim α = α
48 |
49 | ε→0 ε ε→0 ε ε→0 ε ε→0
50 |
51 | Thus, the derivative of f(x) is f′(x) = α, a constant value. This result should also be intuitive—linear functions have a constant rate of change by definition.
52 |
53 | Let’s find the derivative of f(x) = x^2, which appeared in the story. The differential coefficient of f(x) at x = a is
54 |
55 | lim f(a + ε) − f(a) = lim (a + ε)^2 − a^2 = lim 2aε + ε^2 = lim 2a + ε = 2a
56 |
57 | ε→0 ε ε→0 ε ε→0 ε ε→0 ε ε→0
58 |
59 | Thus, the differential coefficient of f(x) at x = a is 2a, or f′(a) = 2a. Therefore, the derivative of f(x) is f′(x) = 2x.
60 |
61 | Summary
62 |
63 | - The calculation of a limit that appears in calculus is simply a formula calculating an error.
64 | - A limit is used to obtain a derivative.
65 | - The derivative is the slope of the tangent line at a given point.
66 | - The derivative is nothing but the rate of change.
--------------------------------------------------------------------------------
/2024/vector-pe-lab/mds/latex-instruction-parsed.md:
--------------------------------------------------------------------------------
1 | # Manga Comic Book
2 |
3 | COMics inside!
4 |
5 | HIROYUKI KOJIMA
6 |
7 | SHIN TOGAMI
8 |
9 | BecomCo, LTD:
10 |
11 | no starch
12 |
13 | press
14 |
15 | Ohmsha
16 | ---
17 | # The Asagake Times
18 |
19 | Sanda-Cho Distributor
20 |
21 | A newspaper distributor?
22 |
23 | do I have the wrong map?
24 |
25 | You're looking for the Sanda-cho branch office?
26 |
27 | It's next door.
28 |
29 | Everybody mistakes us for the office because we are larger.
30 |
31 | What Is a Function? 3
32 | ---
33 | # Derivative Calculation
34 |
35 | Let’s find the derivative of constant function f(x) = α. The differential coefficient of f(x) at x = a is
36 |
37 | lim f(a + ε) − f(a) = lim α − α = lim 0 = 0
38 |
39 | ε→0 ε ε→0 ε ε→0
40 |
41 | Thus, the derivative of f(x) is f′(x) = 0. This makes sense, since our function is constant—the rate of change is 0.
42 |
43 | Note: The differential coefficient of f(x) at x = a is often simply called the derivative of f(x) at x = a, or just f′(a).
44 |
45 | Let’s calculate the derivative of linear function f(x) = αx + β. The derivative of f(x) at x = α is
46 |
47 | lim f(α + ε) − f(a) = lim α(a + ε) + β − (αa + β) = lim α = α
48 |
49 | ε→0 ε ε→0 ε ε→0 ε ε→0
50 |
51 | Thus, the derivative of f(x) is f′(x) = α, a constant value. This result should also be intuitive—linear functions have a constant rate of change by definition.
52 |
53 | Let’s find the derivative of f(x) = x^2, which appeared in the story. The differential coefficient of f(x) at x = a is
54 |
55 | lim f(a + ε) − f(a) = lim (a + ε)^2 − a^2 = lim 2aε + ε^2 = lim 2a + ε = 2a
56 |
57 | ε→0 ε ε→0 ε ε→0 ε ε→0 ε ε→0
58 |
59 | Thus, the differential coefficient of f(x) at x = a is 2a, or f′(a) = 2a. Therefore, the derivative of f(x) is f′(x) = 2x.
60 |
61 | Summary
62 |
63 | - The calculation of a limit that appears in calculus is simply a formula calculating an error.
64 | - A limit is used to obtain a derivative.
65 | - The derivative is the slope of the tangent line at a given point.
66 | - The derivative is nothing but the rate of change.
--------------------------------------------------------------------------------
/2024/vector-pe-lab/mds/parsed.md:
--------------------------------------------------------------------------------
1 | # Guidecomics
2 |
3 | THE MANGA INSIDE!
4 |
5 | CALCULUS
6 |
7 | Hiroyuki KOJIMA
8 |
9 | SHIN TOGAMI
10 |
11 | Becom Co., LTD.
12 |
13 | Ohmsha
14 | ---
15 | The Asagake Times
16 |
17 | Sanda-Cho Distributor
18 |
19 | A newspaper distributor?
20 |
21 | Do I have the wrong map?
22 |
23 | You're looking for the Sanda-cho branch office?
24 |
25 | Everybody mistakes us for the office because we are larger.
26 |
27 | What Is a Function? 3
28 | ---
29 | ## Calculating the Derivative of a Constant, Linear, or Quadratic Function
30 |
31 | |1.|Let’s find the derivative of constant function f(x) = α. The differential coefficient of f(x) at x = a is|
32 | |---|---|
33 | | |lim α → 0 (f(a + ε) - f(a)) / ε = lim α → 0 (α - α) = lim α → 0 0 = 0|
34 | | |Thus, the derivative of f(x) is f′(x) = 0. This makes sense, since our function is constant—the rate of change is 0.|
35 |
36 | Note: The differential coefficient of f(x) at x = a is often simply called the derivative of f(x) at x = a, or just f′(a).
37 |
38 | |2.|Let’s calculate the derivative of linear function f(x) = αx + β. The derivative of f(x) at x = α is|
39 | |---|---|
40 | | |lim ε → 0 (f(α + ε) - f(a)) = lim ε → 0 (α(a + ε) + β - (αa + β)) = lim ε → 0 α = α|
41 | | |Thus, the derivative of f(x) is f′(x) = α, a constant value. This result should also be intuitive—linear functions have a constant rate of change by definition.|
42 |
43 | |3.|Let’s find the derivative of f(x) = x^2, which appeared in the story. The differential coefficient of f(x) at x = a is|
44 | |---|---|
45 | | |lim ε → 0 ((a + ε)^2 - a^2) / ε = lim ε → 0 (a^2 + 2aε + ε^2 - a^2) = lim ε → 0 (2aε + ε^2) = lim ε → 0 (2a + ε) = 2a|
46 | | |Thus, the differential coefficient of f(x) at x = a is 2a, or f′(a) = 2a. Therefore, the derivative of f(x) is f′(x) = 2x.|
47 |
48 | ## Summary
49 |
50 | - The calculation of a limit that appears in calculus is simply a formula calculating an error.
51 | - A limit is used to obtain a derivative.
52 | - The derivative is the slope of the tangent line at a given point.
53 | - The derivative is nothing but the rate of change.
54 |
55 | ## Chapter 1 Let’s Differentiate a Function!
--------------------------------------------------------------------------------
/2024/vector-pe-lab/vector-pe-lab.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "5ab60f84-39b3-4bdd-ae83-6527acb315e5",
6 | "metadata": {},
7 | "source": [
8 | "# LLMs and LlamaIndex ◦ April 4 2024 ◦ Vector Institute Prompt Engineering Lab"
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": null,
14 | "id": "2276edc2-74ce-4d85-bdaa-0607c74fc981",
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "##################################################################\n",
19 | "# Venue: Vector Institute Prompt Engineering Lab\n",
20 | "# Talk: Building RAG Systems with LlamaIndex\n",
21 | "# Speaker: Andrei Fajardo\n",
22 | "##################################################################"
23 | ]
24 | },
25 | {
26 | "cell_type": "markdown",
27 | "id": "3bee89b8-a04d-4326-9392-b9e7e1bcb8af",
28 | "metadata": {},
29 | "source": [
30 | ""
31 | ]
32 | },
33 | {
34 | "cell_type": "markdown",
35 | "id": "e4d38b38-ea48-4012-81ae-84e1d1f40a69",
36 | "metadata": {},
37 | "source": [
38 | ""
39 | ]
40 | },
41 | {
42 | "cell_type": "markdown",
43 | "id": "34d1f8e7-f978-4f19-bdfb-37c0d235b5bf",
44 | "metadata": {},
45 | "source": [
46 | "#### Notebook Setup & Dependency Installation"
47 | ]
48 | },
49 | {
50 | "cell_type": "code",
51 | "execution_count": null,
52 | "id": "f227a52a-a147-4e8f-b7d3-e03f983fd5f1",
53 | "metadata": {
54 | "scrolled": true
55 | },
56 | "outputs": [],
57 | "source": [
58 | "%pip install llama-index-vector-stores-qdrant -q"
59 | ]
60 | },
61 | {
62 | "cell_type": "code",
63 | "execution_count": 1,
64 | "id": "7bc383fc-19b2-47b5-af61-e83210ea9c37",
65 | "metadata": {},
66 | "outputs": [],
67 | "source": [
68 | "import nest_asyncio\n",
69 | "\n",
70 | "nest_asyncio.apply()"
71 | ]
72 | },
73 | {
74 | "cell_type": "code",
75 | "execution_count": 2,
76 | "id": "d56aba9c-230b-464d-909c-de17d1bc0aa6",
77 | "metadata": {},
78 | "outputs": [
79 | {
80 | "name": "stdout",
81 | "output_type": "stream",
82 | "text": [
83 | "mkdir: data: File exists\n",
84 | "--2024-04-04 11:34:26-- https://vectorinstitute.ai/wp-content/uploads/2024/02/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\n",
85 | "Resolving vectorinstitute.ai (vectorinstitute.ai)... 162.159.134.42\n",
86 | "Connecting to vectorinstitute.ai (vectorinstitute.ai)|162.159.134.42|:443... connected.\n",
87 | "HTTP request sent, awaiting response... 200 OK\n",
88 | "Length: 5305903 (5.1M) [application/pdf]\n",
89 | "Saving to: ‘./data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf’\n",
90 | "\n",
91 | "./data/Vector-Annua 100%[===================>] 5.06M 14.2MB/s in 0.4s \n",
92 | "\n",
93 | "2024-04-04 11:34:27 (14.2 MB/s) - ‘./data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf’ saved [5305903/5305903]\n",
94 | "\n"
95 | ]
96 | }
97 | ],
98 | "source": [
99 | "!mkdir data\n",
100 | "!wget \"https://vectorinstitute.ai/wp-content/uploads/2024/02/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\" -O \"./data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\""
101 | ]
102 | },
103 | {
104 | "cell_type": "markdown",
105 | "id": "275c00f1-e358-498a-88c3-8e810a5a2546",
106 | "metadata": {},
107 | "source": [
108 | "## Motivation\n",
109 | "\n",
110 | "\n"
111 | ]
112 | },
113 | {
114 | "cell_type": "code",
115 | "execution_count": 3,
116 | "id": "25d4ce76-8eea-44cb-aa99-94844dfed9c7",
117 | "metadata": {},
118 | "outputs": [],
119 | "source": [
120 | "# query an LLM and ask it about Mistplay\n",
121 | "from llama_index.llms.openai import OpenAI\n",
122 | "\n",
123 | "llm = OpenAI(model=\"gpt-4-turbo-preview\")\n",
124 | "\n",
125 | "response = llm.complete(\n",
126 | " \"What is Vector Institute all about?\"\n",
127 | ")"
128 | ]
129 | },
130 | {
131 | "cell_type": "code",
132 | "execution_count": 4,
133 | "id": "c3f18489-4f25-40ce-86e9-697ddea7d6c6",
134 | "metadata": {},
135 | "outputs": [
136 | {
137 | "name": "stdout",
138 | "output_type": "stream",
139 | "text": [
140 | "The Vector Institute is a research institution in Toronto, Canada, dedicated to the field of artificial intelligence (AI), with a particular focus on machine learning and deep learning. It was established in March 2017 as part of Canada's strategy to lead in the AI domain, leveraging the country's significant contributions to the development of AI technologies.\n",
141 | "\n",
142 | "The institute's mission revolves around several key objectives:\n",
143 | "\n",
144 | "1. **Research Excellence**: Vector aims to attract, retain, and develop top talent in the AI field. It fosters an environment where leading researchers and industry experts can collaborate on cutting-edge projects, pushing the boundaries of what AI can achieve.\n",
145 | "\n",
146 | "2. **Industry Collaboration**: The institute works closely with companies across various sectors, helping them to adopt AI technologies to drive innovation and competitiveness. This includes providing guidance on AI strategy, development, and deployment.\n",
147 | "\n",
148 | "3. **Talent Development**: Recognizing the critical need for skilled professionals in the AI field, Vector offers educational programs and training opportunities. These initiatives are designed to equip students, professionals, and leaders with the knowledge and skills necessary to excel in the rapidly evolving AI landscape.\n",
149 | "\n",
150 | "4. **Thought Leadership**: Vector contributes to the global conversation on AI by sharing insights, research findings, and best practices. It aims to influence policy and decision-making, ensuring that AI development and deployment are guided by ethical considerations and societal needs.\n",
151 | "\n",
152 | "The Vector Institute is part of a broader Canadian AI ecosystem, which includes other leading institutions like the Montreal Institute for Learning Algorithms (MILA) and the Alberta Machine Intelligence Institute (Amii). Together, these organizations help to solidify Canada's position as a world leader in AI research and innovation.\n"
153 | ]
154 | }
155 | ],
156 | "source": [
157 | "print(response)"
158 | ]
159 | },
160 | {
161 | "cell_type": "code",
162 | "execution_count": 5,
163 | "id": "80b2f9c3-1e07-4178-9e7e-220355725955",
164 | "metadata": {},
165 | "outputs": [],
166 | "source": [
167 | "response = llm.complete(\n",
168 | " \"According to Vector Institute's Annual Report 2022-2023, \"\n",
169 | " \"how many AI jobs were created in Ontario?\"\n",
170 | ")"
171 | ]
172 | },
173 | {
174 | "cell_type": "code",
175 | "execution_count": 6,
176 | "id": "6a1c6424-cf2f-4ee1-b601-674b03354e35",
177 | "metadata": {},
178 | "outputs": [
179 | {
180 | "name": "stdout",
181 | "output_type": "stream",
182 | "text": [
183 | "As of my last update in April 2023, I don't have access to the specific details of the Vector Institute's Annual Report for 2022-2023. Therefore, I cannot provide the exact number of AI jobs created in Ontario as reported in that document. For the most accurate and up-to-date information, I recommend checking the Vector Institute's official website or contacting them directly. They may have published the report or related summaries that include the data you're interested in.\n"
184 | ]
185 | }
186 | ],
187 | "source": [
188 | "print(response)"
189 | ]
190 | },
191 | {
192 | "cell_type": "markdown",
193 | "id": "04a0ef8d-d55c-4b64-887b-18d343503a76",
194 | "metadata": {},
195 | "source": [
196 | "## Basic RAG in 3 Steps\n",
197 | "\n",
198 | "\n",
199 | "\n",
200 | "\n",
201 | "1. Build external knowledge (i.e., uploading updated data sources)\n",
202 | "2. Retrieve\n",
203 | "3. Augment and Generate"
204 | ]
205 | },
206 | {
207 | "cell_type": "markdown",
208 | "id": "598a5257-20ae-468e-85d6-d4e8c46b8cb5",
209 | "metadata": {},
210 | "source": [
211 | "## 1. Build External Knowledge\n",
212 | "\n",
213 | ""
214 | ]
215 | },
216 | {
217 | "cell_type": "code",
218 | "execution_count": 7,
219 | "id": "a2963f90-9da5-4a0d-8dbe-f16fcb8627a3",
220 | "metadata": {},
221 | "outputs": [],
222 | "source": [
223 | "\"\"\"Load the data.\n",
224 | "\n",
225 | "With llama-index, before any transformations are applied,\n",
226 | "data is loaded in the `Document` abstraction, which is\n",
227 | "a container that holds the text of the document.\n",
228 | "\"\"\"\n",
229 | "\n",
230 | "from llama_index.core import SimpleDirectoryReader\n",
231 | "\n",
232 | "loader = SimpleDirectoryReader(input_dir=\"./data\")\n",
233 | "documents = loader.load_data()"
234 | ]
235 | },
236 | {
237 | "cell_type": "code",
238 | "execution_count": 8,
239 | "id": "da321e2c-8428-4c04-abf2-b204416e816f",
240 | "metadata": {},
241 | "outputs": [
242 | {
243 | "data": {
244 | "text/plain": [
245 | "'Vector Institute | Annual Report 2022-2023 2\\nTABLE OF CONTENTS\\nMESSAGE FROM THE BOARD CHAIR 3 \\nMESSAGE FROM THE PRESIDENT AND CEO 4\\nVECTOR’S VISION AND MISSION 5\\nONTARIO’S VIBRANT AI ECOSYSTEM 6\\nINDUSTRY INNOVATION 7\\nRESEARCH AND EDUCATION 1 4TALENT AND WORKFORCE DEVELOPMENT 24\\nHEALTH 3 1\\nAI ENGINEERING 36\\nTHOUGHT LEADERSHIP 39\\nFINANCIALS 42\\n'"
246 | ]
247 | },
248 | "execution_count": 8,
249 | "metadata": {},
250 | "output_type": "execute_result"
251 | }
252 | ],
253 | "source": [
254 | "# if you want to see what the text looks like\n",
255 | "documents[1].text"
256 | ]
257 | },
258 | {
259 | "cell_type": "code",
260 | "execution_count": 9,
261 | "id": "4801e74a-8c52-45c4-967d-7a1a94f54ad3",
262 | "metadata": {},
263 | "outputs": [],
264 | "source": [
265 | "\"\"\"Chunk, Encode, and Store into a Vector Store.\n",
266 | "\n",
267 | "To streamline the process, we can make use of the IngestionPipeline\n",
268 | "class that will apply your specified transformations to the\n",
269 | "Document's.\n",
270 | "\"\"\"\n",
271 | "\n",
272 | "from llama_index.core.ingestion import IngestionPipeline\n",
273 | "from llama_index.core.node_parser import SentenceSplitter\n",
274 | "from llama_index.embeddings.openai import OpenAIEmbedding\n",
275 | "from llama_index.vector_stores.qdrant import QdrantVectorStore\n",
276 | "import qdrant_client\n",
277 | "\n",
278 | "client = qdrant_client.QdrantClient(location=\":memory:\")\n",
279 | "vector_store = QdrantVectorStore(client=client, collection_name=\"test_store\")\n",
280 | "\n",
281 | "pipeline = IngestionPipeline(\n",
282 | " transformations = [\n",
283 | " SentenceSplitter(),\n",
284 | " OpenAIEmbedding(),\n",
285 | " ],\n",
286 | " vector_store=vector_store,\n",
287 | ")\n",
288 | "_nodes = pipeline.run(documents=documents, num_workers=4)"
289 | ]
290 | },
291 | {
292 | "cell_type": "code",
293 | "execution_count": 10,
294 | "id": "02afea25-098b-49c7-a965-21c7576757af",
295 | "metadata": {},
296 | "outputs": [
297 | {
298 | "data": {
299 | "text/plain": [
300 | "'Vector Institute | Annual Report 2022-2023 2\\nTABLE OF CONTENTS\\nMESSAGE FROM THE BOARD CHAIR 3 \\nMESSAGE FROM THE PRESIDENT AND CEO 4\\nVECTOR’S VISION AND MISSION 5\\nONTARIO’S VIBRANT AI ECOSYSTEM 6\\nINDUSTRY INNOVATION 7\\nRESEARCH AND EDUCATION 1 4TALENT AND WORKFORCE DEVELOPMENT 24\\nHEALTH 3 1\\nAI ENGINEERING 36\\nTHOUGHT LEADERSHIP 39\\nFINANCIALS 42'"
301 | ]
302 | },
303 | "execution_count": 10,
304 | "metadata": {},
305 | "output_type": "execute_result"
306 | }
307 | ],
308 | "source": [
309 | "# if you want to see the nodes\n",
310 | "# len(_nodes)\n",
311 | "_nodes[1].text"
312 | ]
313 | },
314 | {
315 | "cell_type": "code",
316 | "execution_count": 11,
317 | "id": "44cd8a86-089d-4329-9484-35b98b3a26f9",
318 | "metadata": {},
319 | "outputs": [],
320 | "source": [
321 | "\"\"\"Create a llama-index... wait for it... Index.\n",
322 | "\n",
323 | "After uploading your encoded documents into your vector\n",
324 | "store of choice, you can connect to it with a VectorStoreIndex\n",
325 | "which then gives you access to all of the llama-index functionality.\n",
326 | "\"\"\"\n",
327 | "\n",
328 | "from llama_index.core import VectorStoreIndex\n",
329 | "\n",
330 | "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)"
331 | ]
332 | },
333 | {
334 | "cell_type": "markdown",
335 | "id": "286b1827-7547-49c6-aba3-82f08d6d86b8",
336 | "metadata": {},
337 | "source": [
338 | "## 2. Retrieve Against A Query\n",
339 | "\n",
340 | ""
341 | ]
342 | },
343 | {
344 | "cell_type": "code",
345 | "execution_count": 13,
346 | "id": "49f86af1-db08-4641-89ad-d60abd04e6b3",
347 | "metadata": {},
348 | "outputs": [],
349 | "source": [
350 | "\"\"\"Retrieve relevant documents against a query.\n",
351 | "\n",
352 | "With our Index ready, we can now query it to\n",
353 | "retrieve the most relevant document chunks.\n",
354 | "\"\"\"\n",
355 | "\n",
356 | "retriever = index.as_retriever(similarity_top_k=2)\n",
357 | "retrieved_nodes = retriever.retrieve(\n",
358 | " \"According to Vector Institute's Annual Report 2022-2023, \"\n",
359 | " \"how many AI jobs were created in Ontario?\"\n",
360 | ")"
361 | ]
362 | },
363 | {
364 | "cell_type": "code",
365 | "execution_count": 14,
366 | "id": "06b04a79-2a28-4a21-b15b-4bb6c7b1a227",
367 | "metadata": {},
368 | "outputs": [
369 | {
370 | "data": {
371 | "text/plain": [
372 | "[NodeWithScore(node=TextNode(id_='af5b8613-9990-492f-8a0c-94ced4be3c61', embedding=None, metadata={'page_label': '6', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={: RelatedNodeInfo(node_id='47692fff-7f25-4472-846f-c3f8d30a16e5', node_type=, metadata={'page_label': '6', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='3bbac897823b34c5131c0df0923d9cc6125119814b241c192bb600a50328b432'), : RelatedNodeInfo(node_id='a84bb9b1-771a-40b1-b923-bb5c7d18f02b', node_type=, metadata={'page_label': '5', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='31dd177bdc920cab3187722b550baa45b2b0307b78132444b54c24eb36892c3d'), : RelatedNodeInfo(node_id='b70932b9-b9b2-4094-9e87-9cdc2ed6b90c', node_type=, metadata={}, hash='c8c0e5beea0643c8f86a849ae1c39c181029d182609af9b8dfc1a08d39c9b0b4')}, text='6\\nVector Institute | Annual Report 2022-2023 \\nONTARIO’S VIBRANT AI ECOSYSTEM\\nThe Vector Institute, in collaboration with Deloitte Canada, produces annual snapshots of the \\nOntario AI ecosystem, revealing its continued strength and role in nurturing top AI talent.\\nHIGHLIGHTS OF ONTARIO’S AI ECOSYSTEM\\nMore Ontario data will be available soon on the Vector Institute website . \\n20,634 \\nAI jobs created in Ontario75,975 \\nAI jobs retained in Ontario*1,503 \\nnew AI master’s and study \\npath enrollments1,050\\nnew AI master’s graduates from \\nVector-recognized programs248\\nnew AI-related patents filed\\nacross Canada\\n$1.16 BILLION\\nin AI-related Ontario \\nVC investments312\\ncompanies investe\\nd in the \\nOntario AI ecosystem23% \\nincrease in AI R&D spending or budgets*27\\nnew AI companies established in Ontario68%\\nof CEOs surveyed reported adopting AI solutions*\\n*In this report, Deloitte refined the methodology to improve accuracy of this calculation as well as validated it against other external data sources.', start_char_idx=0, end_char_idx=1009, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.9154188120411539),\n",
373 | " NodeWithScore(node=TextNode(id_='6af722ec-7d4a-40a9-80a0-d0c7919c5fd1', embedding=None, metadata={'page_label': '24', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={: RelatedNodeInfo(node_id='a51fcb51-0caa-44a8-ac08-46c0f9781b28', node_type=, metadata={'page_label': '24', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='2be0f87269fa3f0366e32912eb84d70c2e99facbfb9b15a32a59786d3f04c4dc'), : RelatedNodeInfo(node_id='d10dbc11-c5e3-445e-aa74-73bb7c44f7e5', node_type=, metadata={'page_label': '23', 'file_name': 'Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_path': '/Users/nerdai/talks/2024/vector-pe-lab/data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf', 'file_type': 'application/pdf', 'file_size': 5305903, 'creation_date': '2024-04-04', 'last_modified_date': '2024-02-12'}, hash='efbd640facc631c0c4ff11a69c807b2edf1204b53f3bcb9ca19afad9c7d5abab'), : RelatedNodeInfo(node_id='d21e5d50-1020-4491-987e-fec51f1c2f60', node_type=, metadata={}, hash='1978c85f73b84bc6fdce8b89f5475b65392371816a70973d65b3a6f5fd57e0c1')}, text='24\\nVector Institute | Annual Report 2022-2023 \\nTALENT AND WORKFORCE DEVELOPMENT\\nVECTOR’S CRUCIAL ROLE IN ADVANCING \\nONTARIO’S ECONOMY\\nVector works with both universities and employers to address the \\nincreasing demand for AI expertise by creating channels for promising students to access degree programs that cultivate the most sought-after AI skills in industry. \\nThrough Vector’s initiatives and experiential learning opportunities, \\nstudents learn crucial AI technical and job-related competencies, make connections with leading employers, and contribute to the constructive hiring outcomes that are reshaping Ontario’s economy. \\nAs of March 31, 2023, more than 1,325 students from \\nVector-recognized programs and AI study paths, including Vector Institute Scholarship in Artificial In elligence (VSAI) recipients, have been hired by Ontario employers.\"Ontario is a strong player in the field of artificial \\nintelligence, technology, and research, and we are thrilled to see so many new graduates looking to invest their skills and ideas in the province.\\n\"\\n-Vic Fedeli, Minister of Economic Development, Job Creationand Trade of Ontario*\\n*Quote source', start_char_idx=0, end_char_idx=1164, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.9083415588705721)]"
374 | ]
375 | },
376 | "execution_count": 14,
377 | "metadata": {},
378 | "output_type": "execute_result"
379 | }
380 | ],
381 | "source": [
382 | "# to view the retrieved nodes\n",
383 | "retrieved_nodes"
384 | ]
385 | },
386 | {
387 | "cell_type": "markdown",
388 | "id": "978ae2c5-8c2a-41c7-a2eb-85a5562f2db5",
389 | "metadata": {},
390 | "source": [
391 | "## 3. Generate Final Response\n",
392 | "\n",
393 | ""
394 | ]
395 | },
396 | {
397 | "cell_type": "code",
398 | "execution_count": 15,
399 | "id": "ef33c349-eed4-4e35-9b5d-9473adf2ce01",
400 | "metadata": {},
401 | "outputs": [],
402 | "source": [
403 | "\"\"\"Context-Augemented Generation.\n",
404 | "\n",
405 | "With our Index ready, we can create a QueryEngine\n",
406 | "that handles the retrieval and context augmentation\n",
407 | "in order to get the final response.\n",
408 | "\"\"\"\n",
409 | "\n",
410 | "query_engine = index.as_query_engine()"
411 | ]
412 | },
413 | {
414 | "cell_type": "code",
415 | "execution_count": 16,
416 | "id": "4139c48a-ece8-4244-b4eb-7cff74cb1325",
417 | "metadata": {},
418 | "outputs": [
419 | {
420 | "name": "stdout",
421 | "output_type": "stream",
422 | "text": [
423 | "Context information is below.\n",
424 | "---------------------\n",
425 | "{context_str}\n",
426 | "---------------------\n",
427 | "Given the context information and not prior knowledge, answer the query.\n",
428 | "Query: {query_str}\n",
429 | "Answer: \n"
430 | ]
431 | }
432 | ],
433 | "source": [
434 | "# to inspect the default prompt being used\n",
435 | "print(query_engine.\n",
436 | " get_prompts()[\"response_synthesizer:text_qa_template\"]\n",
437 | " .default_template.template\n",
438 | ")"
439 | ]
440 | },
441 | {
442 | "cell_type": "code",
443 | "execution_count": 17,
444 | "id": "6179639d-af96-4a09-b440-b47ad599a26f",
445 | "metadata": {},
446 | "outputs": [
447 | {
448 | "name": "stdout",
449 | "output_type": "stream",
450 | "text": [
451 | "20,634 AI jobs were created in Ontario according to Vector Institute's Annual Report 2022-2023.\n"
452 | ]
453 | }
454 | ],
455 | "source": [
456 | "response = query_engine.query(\n",
457 | " \"According to Vector Institute's Annual Report 2022-2023, \"\n",
458 | " \"how many AI jobs were created in Ontario?\"\n",
459 | ")\n",
460 | "print(response)"
461 | ]
462 | },
463 | {
464 | "cell_type": "markdown",
465 | "id": "a050bb7f-266c-46c7-8773-eccb909e5fd6",
466 | "metadata": {},
467 | "source": [
468 | "## More Queries"
469 | ]
470 | },
471 | {
472 | "cell_type": "markdown",
473 | "id": "20be9033-f240-4505-9761-29e9455c8974",
474 | "metadata": {},
475 | "source": [
476 | "### Comparisons"
477 | ]
478 | },
479 | {
480 | "cell_type": "code",
481 | "execution_count": 18,
482 | "id": "c86e99c2-601e-443e-aed2-ebe25106a966",
483 | "metadata": {},
484 | "outputs": [
485 | {
486 | "name": "stdout",
487 | "output_type": "stream",
488 | "text": [
489 | "27\n"
490 | ]
491 | }
492 | ],
493 | "source": [
494 | "query = (\n",
495 | " \"According to Vector Institute's 2022-2023 annual report, \"\n",
496 | " \"how many new AI companies were established in Ontario?\"\n",
497 | ")\n",
498 | "\n",
499 | "response = query_engine.query(query)\n",
500 | "print(response)"
501 | ]
502 | },
503 | {
504 | "cell_type": "code",
505 | "execution_count": 19,
506 | "id": "b02d9ae9-42ed-4f6e-8c2f-709d7e97dd22",
507 | "metadata": {},
508 | "outputs": [
509 | {
510 | "name": "stdout",
511 | "output_type": "stream",
512 | "text": [
513 | "In the years 2022 and 2023, the dollar value for Unrestricted net assets was $51,630,471 and $52,937,983, respectively.\n"
514 | ]
515 | }
516 | ],
517 | "source": [
518 | "query = (\n",
519 | " \"According to Vector Institute's 2022-2023 annual report, \"\n",
520 | " \"what was the dollar value for Unrestricted net assets in \"\n",
521 | " \"the years 2022 & 2023?\"\n",
522 | ")\n",
523 | "\n",
524 | "response = query_engine.query(query)\n",
525 | "print(response)"
526 | ]
527 | },
528 | {
529 | "cell_type": "code",
530 | "execution_count": 20,
531 | "id": "5bafbb1e-96e7-4a9f-909f-e7bfffd76532",
532 | "metadata": {},
533 | "outputs": [
534 | {
535 | "name": "stdout",
536 | "output_type": "stream",
537 | "text": [
538 | "According to Vector Institute's 2022-2023 annual report, the platinum sponsors were BMO Financial Group, Google, Loblaw Companies Ltd., NVIDIA, RBC, Scotiabank, Shopify Inc., and TD Bank Group.\n"
539 | ]
540 | }
541 | ],
542 | "source": [
543 | "query = (\n",
544 | " \"According to Vector Institute's 2022-2023 annual report, \"\n",
545 | " \"what companies were platinum sponsors?\"\n",
546 | ")\n",
547 | "\n",
548 | "response = query_engine.query(query)\n",
549 | "print(response)"
550 | ]
551 | },
552 | {
553 | "cell_type": "markdown",
554 | "id": "dae63946-be38-4807-af2a-8113661a806b",
555 | "metadata": {},
556 | "source": [
557 | "## In Summary\n",
558 | "\n",
559 | "- LLMs as powerful as they are, don't perform too well with knowledge-intensive tasks (domain specific, updated data, long-tail)\n",
560 | "- Context augmentation has been shown (in a few studies) to outperform LLMs without augmentation\n",
561 | "- In this notebook, we showed one such example that follows that pattern."
562 | ]
563 | },
564 | {
565 | "cell_type": "markdown",
566 | "id": "53bfd859-924a-4748-b4d8-ee0f2dc52e3b",
567 | "metadata": {},
568 | "source": [
569 | "## Storing Results In Structured Objects"
570 | ]
571 | },
572 | {
573 | "cell_type": "markdown",
574 | "id": "01fb77fd-f4e8-42b0-9704-d2229f4e66a8",
575 | "metadata": {},
576 | "source": [
577 | ""
578 | ]
579 | },
580 | {
581 | "cell_type": "code",
582 | "execution_count": 21,
583 | "id": "a63649cf-7f3a-4f6d-a383-92163f71b03c",
584 | "metadata": {},
585 | "outputs": [],
586 | "source": [
587 | "import json\n",
588 | "from llama_index.core.bridge.pydantic import BaseModel, Field\n",
589 | "from typing import List\n",
590 | "\n",
591 | "from llama_index.program.openai import OpenAIPydanticProgram\n",
592 | "from llama_index.llms.openai import OpenAI"
593 | ]
594 | },
595 | {
596 | "cell_type": "markdown",
597 | "id": "7f306da0-4da6-45b5-b9b3-50d8d0b9883a",
598 | "metadata": {},
599 | "source": [
600 | "### Sponsors"
601 | ]
602 | },
603 | {
604 | "cell_type": "code",
605 | "execution_count": 22,
606 | "id": "b37bc7e7-4ee8-40e8-891f-0307997049a7",
607 | "metadata": {},
608 | "outputs": [],
609 | "source": [
610 | "class VectorSponsors(BaseModel):\n",
611 | " \"\"\"Data model for Vector Institute sponsors 2022-2023.\"\"\"\n",
612 | " \n",
613 | " platinum: str = Field(description=\"Platinum sponsors\")\n",
614 | " gold: str = Field(description=\"Gold sponsors\")\n",
615 | " silver: str = Field(description=\"Silver sponsors\")\n",
616 | " bronze: List[str] = Field(description=\"Bronze sponsors\")"
617 | ]
618 | },
619 | {
620 | "cell_type": "code",
621 | "execution_count": 23,
622 | "id": "ec28d600-5001-4176-bd9f-5982c05d02a3",
623 | "metadata": {},
624 | "outputs": [],
625 | "source": [
626 | "prompt_template_str = \"\"\"\\\n",
627 | "Here is the 2022-2023 Annual Report for Vector Institute:\n",
628 | "{document_text}\n",
629 | "Provide the names sponsor companies according to their tiers.\n",
630 | "\"\"\"\n",
631 | "\n",
632 | "program = OpenAIPydanticProgram.from_defaults(\n",
633 | " output_cls=VectorSponsors,\n",
634 | " prompt_template_str=prompt_template_str,\n",
635 | " llm=OpenAI(\"gpt-4-turbo-preview\"),\n",
636 | " verbose=True\n",
637 | ")"
638 | ]
639 | },
640 | {
641 | "cell_type": "code",
642 | "execution_count": 24,
643 | "id": "bf0b8186-3e5f-4690-a74c-6be28f1f3d74",
644 | "metadata": {},
645 | "outputs": [],
646 | "source": [
647 | "report_as_text = \"\"\n",
648 | "for d in documents:\n",
649 | " report_as_text += d.text"
650 | ]
651 | },
652 | {
653 | "cell_type": "code",
654 | "execution_count": 25,
655 | "id": "0de616a2-c96a-4be0-b8ba-04e537d03e61",
656 | "metadata": {},
657 | "outputs": [
658 | {
659 | "name": "stdout",
660 | "output_type": "stream",
661 | "text": [
662 | "Function call: VectorSponsors with args: {\"platinum\":\"BMO Financial Group, Google, Loblaw Companies Ltd., NVIDIA, RBC, Scotiabank, Shopify Inc., TD Bank Group, Thomson Reuters\",\"gold\":\"Accenture, Air Canada, Bell Canada, Boehringer Ingelheim (Canada) Ltd., Canadian Tire Corporation, Ltd., CIBC, CN, Deloitte Canada, EY Canada, Georgian, KPMG Canada, KT Corporation, Magna International, OMERS, PwC Canada, Roche Canada, Sun Life Financial, TELUS, Thales Canada\",\"silver\":\"EllisDon Corporation, Linamar Corporation\",\"bronze\":[\"Ada\",\"ALS\",\"GoldSpot Discoveries Ltd.\",\"AltaML\",\"Avidbots\",\"BenchSci\",\"Blue J\",\"Canvass Analytics Inc.\",\"Clearpath\",\"Cohere\",\"Cyclica\",\"Darwin AI\",\"Deep Genomics\",\"FreshBooks\",\"Integrate.ai\",\"Layer 6\",\"League\",\"MindBridge Analytics Inc.\",\"Private AI\",\"Riskfuel\",\"Shakudo\",\"Signal 1\",\"Stradigi AI\",\"Surgical Safety Technologies\",\"TealBook\",\"Troj.AI\",\"Wysdom AI\"]}\n"
663 | ]
664 | }
665 | ],
666 | "source": [
667 | "sponsors = program(document_text=report_as_text)"
668 | ]
669 | },
670 | {
671 | "cell_type": "code",
672 | "execution_count": 26,
673 | "id": "41f8b4b0-0f19-4101-9642-1b5bc555dfac",
674 | "metadata": {},
675 | "outputs": [
676 | {
677 | "name": "stdout",
678 | "output_type": "stream",
679 | "text": [
680 | "{\n",
681 | " \"platinum\": \"BMO Financial Group, Google, Loblaw Companies Ltd., NVIDIA, RBC, Scotiabank, Shopify Inc., TD Bank Group, Thomson Reuters\",\n",
682 | " \"gold\": \"Accenture, Air Canada, Bell Canada, Boehringer Ingelheim (Canada) Ltd., Canadian Tire Corporation, Ltd., CIBC, CN, Deloitte Canada, EY Canada, Georgian, KPMG Canada, KT Corporation, Magna International, OMERS, PwC Canada, Roche Canada, Sun Life Financial, TELUS, Thales Canada\",\n",
683 | " \"silver\": \"EllisDon Corporation, Linamar Corporation\",\n",
684 | " \"bronze\": [\n",
685 | " \"Ada\",\n",
686 | " \"ALS\",\n",
687 | " \"GoldSpot Discoveries Ltd.\",\n",
688 | " \"AltaML\",\n",
689 | " \"Avidbots\",\n",
690 | " \"BenchSci\",\n",
691 | " \"Blue J\",\n",
692 | " \"Canvass Analytics Inc.\",\n",
693 | " \"Clearpath\",\n",
694 | " \"Cohere\",\n",
695 | " \"Cyclica\",\n",
696 | " \"Darwin AI\",\n",
697 | " \"Deep Genomics\",\n",
698 | " \"FreshBooks\",\n",
699 | " \"Integrate.ai\",\n",
700 | " \"Layer 6\",\n",
701 | " \"League\",\n",
702 | " \"MindBridge Analytics Inc.\",\n",
703 | " \"Private AI\",\n",
704 | " \"Riskfuel\",\n",
705 | " \"Shakudo\",\n",
706 | " \"Signal 1\",\n",
707 | " \"Stradigi AI\",\n",
708 | " \"Surgical Safety Technologies\",\n",
709 | " \"TealBook\",\n",
710 | " \"Troj.AI\",\n",
711 | " \"Wysdom AI\"\n",
712 | " ]\n",
713 | "}\n"
714 | ]
715 | }
716 | ],
717 | "source": [
718 | "print(json.dumps(sponsors.dict(), indent=4))"
719 | ]
720 | },
721 | {
722 | "cell_type": "markdown",
723 | "id": "17c1c027-be8b-48f4-87ee-06f3e2c71797",
724 | "metadata": {},
725 | "source": [
726 | "### Useful links\n",
727 | "\n",
728 | "[website](https://www.llamaindex.ai/) ◦ [llamahub](https://llamahub.ai) ◦ [llamaparse](https://github.com/run-llama/llama_parse) ◦ [github](https://github.com/run-llama/llama_index) ◦ [medium](https://medium.com/@llama_index) ◦ [rag-bootcamp-poster](https://d3ddy8balm3goa.cloudfront.net/rag-bootcamp-vector/final_poster.excalidraw.svg)"
729 | ]
730 | }
731 | ],
732 | "metadata": {
733 | "kernelspec": {
734 | "display_name": "vector-pe",
735 | "language": "python",
736 | "name": "vector-pe"
737 | },
738 | "language_info": {
739 | "codemirror_mode": {
740 | "name": "ipython",
741 | "version": 3
742 | },
743 | "file_extension": ".py",
744 | "mimetype": "text/x-python",
745 | "name": "python",
746 | "nbconvert_exporter": "python",
747 | "pygments_lexer": "ipython3",
748 | "version": "3.11.3"
749 | }
750 | },
751 | "nbformat": 4,
752 | "nbformat_minor": 5
753 | }
754 |
--------------------------------------------------------------------------------
/2024/vector-rag-bootcamp/rag-bootcamp.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "5ab60f84-39b3-4bdd-ae83-6527acb315e5",
6 | "metadata": {},
7 | "source": [
8 | "# RAG Bootcamp ◦ February 2024 ◦ Vector Institute "
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": 1,
14 | "id": "2276edc2-74ce-4d85-bdaa-0607c74fc981",
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "##################################################################\n",
19 | "# Venue: RAG Bootcamp - Vector Institute Canada\n",
20 | "# Talk: RAG Bootcamp: Intro to RAG with the LlamaIndexFramework\n",
21 | "# Speaker: Andrei Fajardo\n",
22 | "##################################################################"
23 | ]
24 | },
25 | {
26 | "cell_type": "markdown",
27 | "id": "3bee89b8-a04d-4326-9392-b9e7e1bcb8af",
28 | "metadata": {},
29 | "source": [
30 | ""
31 | ]
32 | },
33 | {
34 | "cell_type": "markdown",
35 | "id": "e4d38b38-ea48-4012-81ae-84e1d1f40a69",
36 | "metadata": {},
37 | "source": [
38 | ""
39 | ]
40 | },
41 | {
42 | "cell_type": "markdown",
43 | "id": "34d1f8e7-f978-4f19-bdfb-37c0d235b5bf",
44 | "metadata": {},
45 | "source": [
46 | "#### Notebook Setup & Dependency Installation"
47 | ]
48 | },
49 | {
50 | "cell_type": "code",
51 | "execution_count": 2,
52 | "id": "f227a52a-a147-4e8f-b7d3-e03f983fd5f1",
53 | "metadata": {
54 | "scrolled": true
55 | },
56 | "outputs": [
57 | {
58 | "name": "stdout",
59 | "output_type": "stream",
60 | "text": [
61 | "\n",
62 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
63 | "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.11 -m pip install --upgrade pip\u001b[0m\n",
64 | "Note: you may need to restart the kernel to use updated packages.\n"
65 | ]
66 | }
67 | ],
68 | "source": [
69 | "%pip install llama-index llama-index-vector-stores-qdrant -q"
70 | ]
71 | },
72 | {
73 | "cell_type": "code",
74 | "execution_count": 3,
75 | "id": "7bc383fc-19b2-47b5-af61-e83210ea9c37",
76 | "metadata": {},
77 | "outputs": [],
78 | "source": [
79 | "import nest_asyncio\n",
80 | "\n",
81 | "nest_asyncio.apply()"
82 | ]
83 | },
84 | {
85 | "cell_type": "code",
86 | "execution_count": 4,
87 | "id": "6aa0f82d-bbe0-4487-a9f9-38fa8475edbf",
88 | "metadata": {},
89 | "outputs": [
90 | {
91 | "name": "stdout",
92 | "output_type": "stream",
93 | "text": [
94 | "--2024-02-28 13:05:08-- https://arxiv.org/pdf/2402.09353.pdf\n",
95 | "Resolving arxiv.org (arxiv.org)... 151.101.3.42, 151.101.131.42, 151.101.67.42, ...\n",
96 | "Connecting to arxiv.org (arxiv.org)|151.101.3.42|:443... connected.\n",
97 | "HTTP request sent, awaiting response... 200 OK\n",
98 | "Length: 604050 (590K) [application/pdf]\n",
99 | "Saving to: ‘./data/dorav1.pdf’\n",
100 | "\n",
101 | "./data/dorav1.pdf 100%[===================>] 589.89K --.-KB/s in 0.1s \n",
102 | "\n",
103 | "2024-02-28 13:05:08 (5.23 MB/s) - ‘./data/dorav1.pdf’ saved [604050/604050]\n",
104 | "\n"
105 | ]
106 | }
107 | ],
108 | "source": [
109 | "!mkdir data\n",
110 | "!wget \"https://arxiv.org/pdf/2402.09353.pdf\" -O \"./data/dorav1.pdf\""
111 | ]
112 | },
113 | {
114 | "cell_type": "markdown",
115 | "id": "275c00f1-e358-498a-88c3-8e810a5a2546",
116 | "metadata": {},
117 | "source": [
118 | "## Motivation\n",
119 | "\n",
120 | "\n"
121 | ]
122 | },
123 | {
124 | "cell_type": "code",
125 | "execution_count": 5,
126 | "id": "25d4ce76-8eea-44cb-aa99-94844dfed9c7",
127 | "metadata": {},
128 | "outputs": [],
129 | "source": [
130 | "# query an LLM and ask it about DoRA\n",
131 | "from llama_index.llms.openai import OpenAI\n",
132 | "\n",
133 | "llm = OpenAI(model=\"gpt-4\")\n",
134 | "response = llm.complete(\"What is DoRA?\")"
135 | ]
136 | },
137 | {
138 | "cell_type": "code",
139 | "execution_count": 6,
140 | "id": "c3f18489-4f25-40ce-86e9-697ddea7d6c6",
141 | "metadata": {},
142 | "outputs": [
143 | {
144 | "name": "stdout",
145 | "output_type": "stream",
146 | "text": [
147 | "Without specific context, it's hard to determine which \"DoRA\" you're referring to as it could mean different things in different fields. However, in the context of education, DoRA often stands for \"Division of Research Administration\" which is responsible for the oversight of research activities in an institution. Please provide more context.\n"
148 | ]
149 | }
150 | ],
151 | "source": [
152 | "print(response.text)"
153 | ]
154 | },
155 | {
156 | "cell_type": "markdown",
157 | "id": "04a0ef8d-d55c-4b64-887b-18d343503a76",
158 | "metadata": {},
159 | "source": [
160 | "## Basic RAG in 3 Steps\n",
161 | "\n",
162 | "\n",
163 | "\n",
164 | "\n",
165 | "1. Build external knowledge (i.e., uploading updated data sources)\n",
166 | "2. Retrieve\n",
167 | "3. Augment and Generate"
168 | ]
169 | },
170 | {
171 | "cell_type": "markdown",
172 | "id": "598a5257-20ae-468e-85d6-d4e8c46b8cb5",
173 | "metadata": {},
174 | "source": [
175 | "## 1. Build External Knowledge\n",
176 | "\n",
177 | ""
178 | ]
179 | },
180 | {
181 | "cell_type": "code",
182 | "execution_count": 7,
183 | "id": "a2963f90-9da5-4a0d-8dbe-f16fcb8627a3",
184 | "metadata": {},
185 | "outputs": [],
186 | "source": [
187 | "\"\"\"Load the data.\n",
188 | "\n",
189 | "With llama-index, before any transformations are applied,\n",
190 | "data is loaded in the `Document` abstraction, which is\n",
191 | "a container that holds the text of the document.\n",
192 | "\"\"\"\n",
193 | "\n",
194 | "from llama_index.core import SimpleDirectoryReader\n",
195 | "\n",
196 | "loader = SimpleDirectoryReader(input_dir=\"./data\")\n",
197 | "documents = loader.load_data()"
198 | ]
199 | },
200 | {
201 | "cell_type": "code",
202 | "execution_count": 8,
203 | "id": "da321e2c-8428-4c04-abf2-b204416e816f",
204 | "metadata": {},
205 | "outputs": [
206 | {
207 | "data": {
208 | "text/plain": [
209 | "'DoRA: Weight-Decomposed Low-Rank Adaptation\\nShih-Yang Liu* 1 2Chien-Yi Wang1Hongxu Yin1Pavlo Molchanov1Yu-Chiang Frank Wang1\\nKwang-Ting Cheng2Min-Hung Chen1\\nAbstract\\nAmong the widely used parameter-efficient fine-\\ntuning (PEFT) methods, LoRA and its variants\\nhave gained considerable popularity because of\\navoiding additional inference costs. However,\\nthere still often exists an accuracy gap between\\nthese methods and full fine-tuning (FT). In this\\nwork, we first introduce a novel weight decom-\\nposition analysis to investigate the inherent dif-\\nferences between FT and LoRA. Aiming to re-\\nsemble the learning capacity of FT from the\\nfindings, we propose Weight- Decomposed L ow-\\nRankAdaptation ( DoRA ). DoRA decomposes\\nthe pre-trained weight into two components, mag-\\nnitude anddirection , for fine-tuning, specifically\\nemploying LoRA for directional updates to effi-\\nciently minimize the number of trainable param-\\neters. By employing DoRA, we enhance both\\nthe learning capacity and training stability of\\nLoRA while avoiding any additional inference\\noverhead. DoRA consistently outperforms LoRA\\non fine-tuning LLaMA, LLaV A, and VL-BART\\non various downstream tasks, such as common-\\nsense reasoning, visual instruction tuning, and\\nimage/video-text understanding. We will release\\nthe code and models upon acceptance.\\n1. Introduction\\nModels that are pre-trained with extensive general domain\\ndatasets have demonstrated remarkable generalization abil-\\nities, significantly benefiting a wide array of applications,\\nfrom natural language processing (NLP) tasks (Qin et al.,\\n2023; Taori et al., 2023) to multi-modal tasks (Li et al., 2022;\\nLiu et al., 2023a). To tailor these general models for spe-\\ncific downstream tasks, full fine-tuning (FT) is commonly\\nemployed, involving the retraining of all model parameters.\\nNevertheless, as the size of models and datasets expand\\n*Work done during an internship at NVIDIA\\n1NVIDIA2HKUST. Correspondence to: Min-Hung Chen\\n.\\nMagnitude\\nB\\nAPretrained\\nWeight\\nMerged\\nWeightPretrained\\nWeight\\nPretrained\\nWeight\\nAdaptFrozen\\nTrainable\\nMagnitude\\nDirection DirectionDecompose\\n(Initialize)MergeFigure 1. An overview of our proposed DoRA, which decomposes\\nthe pre-trained weight into magnitude anddirection components\\nfor fine-tuning, especially with LoRA to efficiently update the\\ndirection component. Note that || · || cdenotes the vector-wise\\nnorm of a matrix across each column vector.\\nin scale, the expense associated with fine-tuning the entire\\nmodel becomes prohibitively large.\\nTo address this issue, parameter-efficient fine-tuning (PEFT)\\nmethods (Houlsby et al., 2019) have been introduced to fine-\\ntune the pre-trained models with only a minimal number of\\nparameters. Among these, LoRA (Hu et al., 2022), which\\ndoes not change the model architecture, has become notably\\npopular for its simplicity and efficacy. Nevertheless, there\\nis still a capacity gap between LoRA and FT, which is often\\nattributed to the limited number of trainable parameters\\nwithout further exploration of other underlying causes (Hu\\net al., 2022; Kopiczko et al., 2024).\\nDrawing on Weight Normalization (Salimans & Kingma,\\n2016), which achieves faster convergence via improving the\\nconditioning of the gradient with weight reparameterization,\\nwe introduce a novel weight decomposition analysis that ini-\\ntially reparameterizes model weights into magnitude and di-\\nrectional components, subsequently examining the changes\\nin magnitude and direction introduced by LoRA and FT.\\nOur analysis reveals that LoRA and FT exhibit markedly\\n1arXiv:2402.09353v1 [cs.CL] 14 Feb 2024'"
210 | ]
211 | },
212 | "execution_count": 8,
213 | "metadata": {},
214 | "output_type": "execute_result"
215 | }
216 | ],
217 | "source": [
218 | "# if you want to see what the text looks like\n",
219 | "documents[0].text"
220 | ]
221 | },
222 | {
223 | "cell_type": "code",
224 | "execution_count": 9,
225 | "id": "4801e74a-8c52-45c4-967d-7a1a94f54ad3",
226 | "metadata": {},
227 | "outputs": [],
228 | "source": [
229 | "\"\"\"Chunk, Encode, and Store into a Vector Store.\n",
230 | "\n",
231 | "To streamline the process, we can make use of the IngestionPipeline\n",
232 | "class that will apply your specified transformations to the\n",
233 | "Document's.\n",
234 | "\"\"\"\n",
235 | "\n",
236 | "from llama_index.core.ingestion import IngestionPipeline\n",
237 | "from llama_index.core.node_parser import SentenceSplitter\n",
238 | "from llama_index.embeddings.openai import OpenAIEmbedding\n",
239 | "from llama_index.vector_stores.qdrant import QdrantVectorStore\n",
240 | "import qdrant_client\n",
241 | "\n",
242 | "client = qdrant_client.QdrantClient(location=\":memory:\")\n",
243 | "vector_store = QdrantVectorStore(client=client, collection_name=\"test_store\")\n",
244 | "\n",
245 | "pipeline = IngestionPipeline(\n",
246 | " transformations = [\n",
247 | " SentenceSplitter(),\n",
248 | " OpenAIEmbedding(),\n",
249 | " ],\n",
250 | " vector_store=vector_store,\n",
251 | ")\n",
252 | "_nodes = pipeline.run(documents=documents, num_workers=4)"
253 | ]
254 | },
255 | {
256 | "cell_type": "code",
257 | "execution_count": 12,
258 | "id": "02afea25-098b-49c7-a965-21c7576757af",
259 | "metadata": {},
260 | "outputs": [],
261 | "source": [
262 | "# if you want to see the nodes\n",
263 | "# len(_nodes)\n",
264 | "# _nodes[0].text"
265 | ]
266 | },
267 | {
268 | "cell_type": "code",
269 | "execution_count": 13,
270 | "id": "44cd8a86-089d-4329-9484-35b98b3a26f9",
271 | "metadata": {},
272 | "outputs": [],
273 | "source": [
274 | "\"\"\"Create a llama-index... wait for it... Index.\n",
275 | "\n",
276 | "After uploading your encoded documents into your vector\n",
277 | "store of choice, you can connect to it with a VectorStoreIndex\n",
278 | "which then gives you access to all of the llama-index functionality.\n",
279 | "\"\"\"\n",
280 | "\n",
281 | "from llama_index.core import VectorStoreIndex\n",
282 | "\n",
283 | "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)"
284 | ]
285 | },
286 | {
287 | "cell_type": "markdown",
288 | "id": "286b1827-7547-49c6-aba3-82f08d6d86b8",
289 | "metadata": {},
290 | "source": [
291 | "## 2. Retrieve Against A Query\n",
292 | "\n",
293 | ""
294 | ]
295 | },
296 | {
297 | "cell_type": "code",
298 | "execution_count": 14,
299 | "id": "49f86af1-db08-4641-89ad-d60abd04e6b3",
300 | "metadata": {},
301 | "outputs": [],
302 | "source": [
303 | "\"\"\"Retrieve relevant documents against a query.\n",
304 | "\n",
305 | "With our Index ready, we can now query it to\n",
306 | "retrieve the most relevant document chunks.\n",
307 | "\"\"\"\n",
308 | "\n",
309 | "retriever = index.as_retriever(similarity_top_k=2)\n",
310 | "retrieved_nodes = retriever.retrieve(\"What is DoRA?\")"
311 | ]
312 | },
313 | {
314 | "cell_type": "code",
315 | "execution_count": 17,
316 | "id": "05f9ce3b-a4e3-4862-b58c-2d9fba1f9abc",
317 | "metadata": {},
318 | "outputs": [],
319 | "source": [
320 | "# to view the retrieved node\n",
321 | "# print(retrieved_nodes[0].text)"
322 | ]
323 | },
324 | {
325 | "cell_type": "markdown",
326 | "id": "978ae2c5-8c2a-41c7-a2eb-85a5562f2db5",
327 | "metadata": {},
328 | "source": [
329 | "## 3. Generate Final Response\n",
330 | "\n",
331 | ""
332 | ]
333 | },
334 | {
335 | "cell_type": "code",
336 | "execution_count": 18,
337 | "id": "ef33c349-eed4-4e35-9b5d-9473adf2ce01",
338 | "metadata": {},
339 | "outputs": [],
340 | "source": [
341 | "\"\"\"Context-Augemented Generation.\n",
342 | "\n",
343 | "With our Index ready, we can create a QueryEngine\n",
344 | "that handles the retrieval and context augmentation\n",
345 | "in order to get the final response.\n",
346 | "\"\"\"\n",
347 | "\n",
348 | "query_engine = index.as_query_engine()"
349 | ]
350 | },
351 | {
352 | "cell_type": "code",
353 | "execution_count": 19,
354 | "id": "4139c48a-ece8-4244-b4eb-7cff74cb1325",
355 | "metadata": {},
356 | "outputs": [
357 | {
358 | "name": "stdout",
359 | "output_type": "stream",
360 | "text": [
361 | "Context information is below.\n",
362 | "---------------------\n",
363 | "{context_str}\n",
364 | "---------------------\n",
365 | "Given the context information and not prior knowledge, answer the query.\n",
366 | "Query: {query_str}\n",
367 | "Answer: \n"
368 | ]
369 | }
370 | ],
371 | "source": [
372 | "# to inspect the default prompt being used\n",
373 | "print(query_engine.\n",
374 | " get_prompts()[\"response_synthesizer:text_qa_template\"]\n",
375 | " .default_template.template\n",
376 | ")"
377 | ]
378 | },
379 | {
380 | "cell_type": "code",
381 | "execution_count": 20,
382 | "id": "6179639d-af96-4a09-b440-b47ad599a26f",
383 | "metadata": {},
384 | "outputs": [
385 | {
386 | "name": "stdout",
387 | "output_type": "stream",
388 | "text": [
389 | "DoRA is a method that enhances the learning capacity of LoRA by introducing incremental directional updates that can be adapted by different LoRA variants. It can replace the concept of incremental directional updates with alternative LoRA variants, such as VeRA, which suggests freezing a unique pair of random low-rank matrices shared across all layers and using minimal layer-specific trainable scaling vectors to capture each layer's incremental updates. This approach significantly reduces the number of trainable parameters while maintaining accuracy. DoRA has been shown to consistently outperform LoRA across various rank settings for commonsense reasoning tasks, indicating its effectiveness in improving accuracy with fewer trainable parameters.\n"
390 | ]
391 | }
392 | ],
393 | "source": [
394 | "response = query_engine.query(\"What is DoRA?\")\n",
395 | "print(response)"
396 | ]
397 | },
398 | {
399 | "cell_type": "markdown",
400 | "id": "dae63946-be38-4807-af2a-8113661a806b",
401 | "metadata": {},
402 | "source": [
403 | "## In Summary\n",
404 | "\n",
405 | "- LLMs as powerful as they are, don't perform too well with knowledge-intensive tasks (domain specific, updated data, long-tail)\n",
406 | "- Context augmentation has been shown (in a few studies) to outperform LLMs without augmentation\n",
407 | "- In this notebook, we showed one such example that follows that pattern."
408 | ]
409 | },
410 | {
411 | "cell_type": "markdown",
412 | "id": "fc857227-3fed-4bb6-a062-99ea3c55e294",
413 | "metadata": {},
414 | "source": [
415 | "# LlamaIndex Has More To Offer\n",
416 | "\n",
417 | "- Data infrastructure that enables production-grade, advanced RAG systems\n",
418 | "- Agentic solutions\n",
419 | "- Newly released: `llama-index-networks`\n",
420 | "- Enterprise offerings (alpha):\n",
421 | " - LlamaParse (proprietary complex PDF parser) and\n",
422 | " - LlamaCloud"
423 | ]
424 | },
425 | {
426 | "cell_type": "markdown",
427 | "id": "17c1c027-be8b-48f4-87ee-06f3e2c71797",
428 | "metadata": {},
429 | "source": [
430 | "### Useful links\n",
431 | "\n",
432 | "[website](https://www.llamaindex.ai/) ◦ [llamahub](https://llamahub.ai) ◦ [github](https://github.com/run-llama/llama_index) ◦ [medium](https://medium.com/@llama_index) ◦ [rag-bootcamp-poster](https://d3ddy8balm3goa.cloudfront.net/rag-bootcamp-vector/final_poster.excalidraw.svg)"
433 | ]
434 | },
435 | {
436 | "cell_type": "code",
437 | "execution_count": null,
438 | "id": "4876f95e-0a1c-4fcf-b155-c8ed688f6649",
439 | "metadata": {},
440 | "outputs": [],
441 | "source": []
442 | }
443 | ],
444 | "metadata": {
445 | "kernelspec": {
446 | "display_name": "rag-bootcamp",
447 | "language": "python",
448 | "name": "rag-bootcamp"
449 | },
450 | "language_info": {
451 | "codemirror_mode": {
452 | "name": "ipython",
453 | "version": 3
454 | },
455 | "file_extension": ".py",
456 | "mimetype": "text/x-python",
457 | "name": "python",
458 | "nbconvert_exporter": "python",
459 | "pygments_lexer": "ipython3",
460 | "version": "3.11.3"
461 | }
462 | },
463 | "nbformat": 4,
464 | "nbformat_minor": 5
465 | }
466 |
--------------------------------------------------------------------------------