├── vid_screenshot.png ├── local_agent_diagram.png ├── README.md ├── LICENSE └── llama3_research_agent.ipynb /vid_screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ALucek/llama3-websearch-agent/main/vid_screenshot.png -------------------------------------------------------------------------------- /local_agent_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ALucek/llama3-websearch-agent/main/local_agent_diagram.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Code from my Llama 3 8b Web Research Agent Video! 2 | 3 | Note! If you're getting error code `202` with DuckDuckGo_Search, you need to downgrade to a specific package. Run `pip install -U duckduckgo_search==5.3.0b4` 4 | 5 | [![s2s](vid_screenshot.png)](https://youtu.be/9K51Leyv3qI) 6 | Click play to watch :) 7 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Adam Łucek 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /llama3_research_agent.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0c707a92-6f82-44d8-8de0-fa612064df5e", 6 | "metadata": {}, 7 | "source": [ 8 | "# Local Web Research Agent w/ Llama 3 8b\n", 9 | "\n", 10 | "### [Llama 3 Release](https://llama.meta.com/llama3/)\n", 11 | "\n", 12 | "### [Ollama Llama 3 Model](https://ollama.com/library/llama3)\n", 13 | "---\n", 14 | "\n", 15 | "![diagram](local_agent_diagram.png)" 16 | ] 17 | }, 18 | { 19 | "cell_type": "markdown", 20 | "id": "715059b3-857c-456d-a740-24e2551d739d", 21 | "metadata": {}, 22 | "source": [ 23 | "---\n", 24 | "[Llama 3 Prompt Format](https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/)\n", 25 | "\n", 26 | "### Special Tokens used with Meta Llama 3\n", 27 | "* **<|begin_of_text|>**: This is equivalent to the BOS token\n", 28 | "* **<|eot_id|>**: This signifies the end of the message in a turn.\n", 29 | "* **<|start_header_id|>{role}<|end_header_id|>**: These tokens enclose the role for a particular message. The possible roles can be: system, user, assistant.\n", 30 | "* **<|end_of_text|>**: This is equivalent to the EOS token. On generating this token, Llama 3 will cease to generate more tokens.\n", 31 | "A prompt should contain a single system message, can contain multiple alternating user and assistant messages, and always ends with the last user message followed by the assistant header." 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": 1, 37 | "id": "35f2cb84-6abf-4a6c-8d1f-cdc6474b77ee", 38 | "metadata": {}, 39 | "outputs": [], 40 | "source": [ 41 | "# Displaying final output format\n", 42 | "from IPython.display import display, Markdown, Latex\n", 43 | "# LangChain Dependencies\n", 44 | "from langchain.prompts import PromptTemplate\n", 45 | "from langchain_core.output_parsers import JsonOutputParser, StrOutputParser\n", 46 | "from langchain_community.chat_models import ChatOllama\n", 47 | "from langchain_community.tools import DuckDuckGoSearchRun\n", 48 | "from langchain_community.utilities import DuckDuckGoSearchAPIWrapper\n", 49 | "from langgraph.graph import END, StateGraph\n", 50 | "# For State Graph \n", 51 | "from typing_extensions import TypedDict\n", 52 | "import os" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 2, 58 | "id": "d39b8539-1bfe-4001-b7b2-6752a77846d5", 59 | "metadata": {}, 60 | "outputs": [], 61 | "source": [ 62 | "# Environment Variables\n", 63 | "os.environ['LANGCHAIN_TRACING_V2'] = 'true'\n", 64 | "os.environ[\"LANGCHAIN_PROJECT\"] = \"L3 Research Agent\"" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": 3, 70 | "id": "9b341d1d-0a59-4c03-8558-759ea00171bb", 71 | "metadata": {}, 72 | "outputs": [], 73 | "source": [ 74 | "# Defining LLM\n", 75 | "local_llm = 'llama3'\n", 76 | "llama3 = ChatOllama(model=local_llm, temperature=0)\n", 77 | "llama3_json = ChatOllama(model=local_llm, format='json', temperature=0)" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 4, 83 | "id": "4c7813ac-791f-4035-a5ec-04810d5de5f9", 84 | "metadata": { 85 | "scrolled": true 86 | }, 87 | "outputs": [], 88 | "source": [ 89 | "# Web Search Tool\n", 90 | "\n", 91 | "wrapper = DuckDuckGoSearchAPIWrapper(max_results=25)\n", 92 | "web_search_tool = DuckDuckGoSearchRun(api_wrapper=wrapper)\n", 93 | "\n", 94 | "# Test Run\n", 95 | "# resp = web_search_tool.invoke(\"home depot news\")\n", 96 | "# resp" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": 5, 102 | "id": "2d798a81-6ed6-4a4f-a1d9-93b4e3059fee", 103 | "metadata": {}, 104 | "outputs": [], 105 | "source": [ 106 | "# Generation Prompt\n", 107 | "\n", 108 | "generate_prompt = PromptTemplate(\n", 109 | " template=\"\"\"\n", 110 | " \n", 111 | " <|begin_of_text|>\n", 112 | " \n", 113 | " <|start_header_id|>system<|end_header_id|> \n", 114 | " \n", 115 | " You are an AI assistant for Research Question Tasks, that synthesizes web search results. \n", 116 | " Strictly use the following pieces of web search context to answer the question. If you don't know the answer, just say that you don't know. \n", 117 | " keep the answer concise, but provide all of the details you can in the form of a research report. \n", 118 | " Only make direct references to material if provided in the context.\n", 119 | " \n", 120 | " <|eot_id|>\n", 121 | " \n", 122 | " <|start_header_id|>user<|end_header_id|>\n", 123 | " \n", 124 | " Question: {question} \n", 125 | " Web Search Context: {context} \n", 126 | " Answer: \n", 127 | " \n", 128 | " <|eot_id|>\n", 129 | " \n", 130 | " <|start_header_id|>assistant<|end_header_id|>\"\"\",\n", 131 | " input_variables=[\"question\", \"context\"],\n", 132 | ")\n", 133 | "\n", 134 | "# Chain\n", 135 | "generate_chain = generate_prompt | llama3 | StrOutputParser()\n", 136 | "\n", 137 | "# Test Run\n", 138 | "# question = \"How are you?\"\n", 139 | "# context = \"\"\n", 140 | "# generation = generate_chain.invoke({\"context\": context, \"question\": question})\n", 141 | "# print(generation)" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": 13, 147 | "id": "49fa1965-6bd4-4dfc-9eb8-96c6cff7b639", 148 | "metadata": {}, 149 | "outputs": [ 150 | { 151 | "name": "stdout", 152 | "output_type": "stream", 153 | "text": [ 154 | "{'choice': 'generate'}\n" 155 | ] 156 | } 157 | ], 158 | "source": [ 159 | "# Router\n", 160 | "\n", 161 | "router_prompt = PromptTemplate(\n", 162 | " template=\"\"\"\n", 163 | " \n", 164 | " <|begin_of_text|>\n", 165 | " \n", 166 | " <|start_header_id|>system<|end_header_id|>\n", 167 | " \n", 168 | " You are an expert at routing a user question to either the generation stage or web search. \n", 169 | " Use the web search for questions that require more context for a better answer, or recent events.\n", 170 | " Otherwise, you can skip and go straight to the generation phase to respond.\n", 171 | " You do not need to be stringent with the keywords in the question related to these topics.\n", 172 | " Give a binary choice 'web_search' or 'generate' based on the question. \n", 173 | " Return the JSON with a single key 'choice' with no premable or explanation. \n", 174 | " \n", 175 | " Question to route: {question} \n", 176 | " \n", 177 | " <|eot_id|>\n", 178 | " \n", 179 | " <|start_header_id|>assistant<|end_header_id|>\n", 180 | " \n", 181 | " \"\"\",\n", 182 | " input_variables=[\"question\"],\n", 183 | ")\n", 184 | "\n", 185 | "# Chain\n", 186 | "question_router = router_prompt | llama3_json | JsonOutputParser()\n", 187 | "\n", 188 | "# Test Run\n", 189 | "question = \"What's up?\"\n", 190 | "print(question_router.invoke({\"question\": question}))" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": 14, 196 | "id": "33ab4128-e0b0-4f49-9f36-1d3bf5636715", 197 | "metadata": {}, 198 | "outputs": [ 199 | { 200 | "name": "stdout", 201 | "output_type": "stream", 202 | "text": [ 203 | "{'query': 'Macom recent news'}\n" 204 | ] 205 | } 206 | ], 207 | "source": [ 208 | "# Query Transformation\n", 209 | "\n", 210 | "query_prompt = PromptTemplate(\n", 211 | " template=\"\"\"\n", 212 | " \n", 213 | " <|begin_of_text|>\n", 214 | " \n", 215 | " <|start_header_id|>system<|end_header_id|> \n", 216 | " \n", 217 | " You are an expert at crafting web search queries for research questions.\n", 218 | " More often than not, a user will ask a basic question that they wish to learn more about, however it might not be in the best format. \n", 219 | " Reword their query to be the most effective web search string possible.\n", 220 | " Return the JSON with a single key 'query' with no premable or explanation. \n", 221 | " \n", 222 | " Question to transform: {question} \n", 223 | " \n", 224 | " <|eot_id|>\n", 225 | " \n", 226 | " <|start_header_id|>assistant<|end_header_id|>\n", 227 | " \n", 228 | " \"\"\",\n", 229 | " input_variables=[\"question\"],\n", 230 | ")\n", 231 | "\n", 232 | "# Chain\n", 233 | "query_chain = query_prompt | llama3_json | JsonOutputParser()\n", 234 | "\n", 235 | "# Test Run\n", 236 | "question = \"What's happened recently with Macom?\"\n", 237 | "print(query_chain.invoke({\"question\": question}))" 238 | ] 239 | }, 240 | { 241 | "cell_type": "code", 242 | "execution_count": 8, 243 | "id": "a1c8e922-3f00-48d6-83cb-cc78a2292838", 244 | "metadata": {}, 245 | "outputs": [], 246 | "source": [ 247 | "# Graph State\n", 248 | "class GraphState(TypedDict):\n", 249 | " \"\"\"\n", 250 | " Represents the state of our graph.\n", 251 | "\n", 252 | " Attributes:\n", 253 | " question: question\n", 254 | " generation: LLM generation\n", 255 | " search_query: revised question for web search\n", 256 | " context: web_search result\n", 257 | " \"\"\"\n", 258 | " question : str\n", 259 | " generation : str\n", 260 | " search_query : str\n", 261 | " context : str\n", 262 | "\n", 263 | "# Node - Generate\n", 264 | "\n", 265 | "def generate(state):\n", 266 | " \"\"\"\n", 267 | " Generate answer\n", 268 | "\n", 269 | " Args:\n", 270 | " state (dict): The current graph state\n", 271 | "\n", 272 | " Returns:\n", 273 | " state (dict): New key added to state, generation, that contains LLM generation\n", 274 | " \"\"\"\n", 275 | " \n", 276 | " print(\"Step: Generating Final Response\")\n", 277 | " question = state[\"question\"]\n", 278 | " context = state[\"context\"]\n", 279 | "\n", 280 | " # Answer Generation\n", 281 | " generation = generate_chain.invoke({\"context\": context, \"question\": question})\n", 282 | " return {\"generation\": generation}\n", 283 | "\n", 284 | "# Node - Query Transformation\n", 285 | "\n", 286 | "def transform_query(state):\n", 287 | " \"\"\"\n", 288 | " Transform user question to web search\n", 289 | "\n", 290 | " Args:\n", 291 | " state (dict): The current graph state\n", 292 | "\n", 293 | " Returns:\n", 294 | " state (dict): Appended search query\n", 295 | " \"\"\"\n", 296 | " \n", 297 | " print(\"Step: Optimizing Query for Web Search\")\n", 298 | " question = state['question']\n", 299 | " gen_query = query_chain.invoke({\"question\": question})\n", 300 | " search_query = gen_query[\"query\"]\n", 301 | " return {\"search_query\": search_query}\n", 302 | "\n", 303 | "\n", 304 | "# Node - Web Search\n", 305 | "\n", 306 | "def web_search(state):\n", 307 | " \"\"\"\n", 308 | " Web search based on the question\n", 309 | "\n", 310 | " Args:\n", 311 | " state (dict): The current graph state\n", 312 | "\n", 313 | " Returns:\n", 314 | " state (dict): Appended web results to context\n", 315 | " \"\"\"\n", 316 | "\n", 317 | " search_query = state['search_query']\n", 318 | " print(f'Step: Searching the Web for: \"{search_query}\"')\n", 319 | " \n", 320 | " # Web search tool call\n", 321 | " search_result = web_search_tool.invoke(search_query)\n", 322 | " return {\"context\": search_result}\n", 323 | "\n", 324 | "\n", 325 | "# Conditional Edge, Routing\n", 326 | "\n", 327 | "def route_question(state):\n", 328 | " \"\"\"\n", 329 | " route question to web search or generation.\n", 330 | "\n", 331 | " Args:\n", 332 | " state (dict): The current graph state\n", 333 | "\n", 334 | " Returns:\n", 335 | " str: Next node to call\n", 336 | " \"\"\"\n", 337 | "\n", 338 | " print(\"Step: Routing Query\")\n", 339 | " question = state['question']\n", 340 | " output = question_router.invoke({\"question\": question})\n", 341 | " if output['choice'] == \"web_search\":\n", 342 | " print(\"Step: Routing Query to Web Search\")\n", 343 | " return \"websearch\"\n", 344 | " elif output['choice'] == 'generate':\n", 345 | " print(\"Step: Routing Query to Generation\")\n", 346 | " return \"generate\"" 347 | ] 348 | }, 349 | { 350 | "cell_type": "code", 351 | "execution_count": 9, 352 | "id": "8f665713-e80b-4d86-8015-77ba55506004", 353 | "metadata": {}, 354 | "outputs": [], 355 | "source": [ 356 | "# Build the nodes\n", 357 | "workflow = StateGraph(GraphState)\n", 358 | "workflow.add_node(\"websearch\", web_search)\n", 359 | "workflow.add_node(\"transform_query\", transform_query)\n", 360 | "workflow.add_node(\"generate\", generate)\n", 361 | "\n", 362 | "# Build the edges\n", 363 | "workflow.set_conditional_entry_point(\n", 364 | " route_question,\n", 365 | " {\n", 366 | " \"websearch\": \"transform_query\",\n", 367 | " \"generate\": \"generate\",\n", 368 | " },\n", 369 | ")\n", 370 | "workflow.add_edge(\"transform_query\", \"websearch\")\n", 371 | "workflow.add_edge(\"websearch\", \"generate\")\n", 372 | "workflow.add_edge(\"generate\", END)\n", 373 | "\n", 374 | "# Compile the workflow\n", 375 | "local_agent = workflow.compile()" 376 | ] 377 | }, 378 | { 379 | "cell_type": "code", 380 | "execution_count": 10, 381 | "id": "4f53aa05-20b2-420e-9a8f-bf12b1e547ab", 382 | "metadata": {}, 383 | "outputs": [], 384 | "source": [ 385 | "def run_agent(query):\n", 386 | " output = local_agent.invoke({\"question\": query})\n", 387 | " print(\"=======\")\n", 388 | " display(Markdown(output[\"generation\"]))" 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": 11, 394 | "id": "6a1b135d-131e-4276-b40c-12ea8b78c39c", 395 | "metadata": {}, 396 | "outputs": [ 397 | { 398 | "name": "stdout", 399 | "output_type": "stream", 400 | "text": [ 401 | "Step: Routing Query\n", 402 | "Step: Routing Query to Web Search\n", 403 | "Step: Optimizing Query for Web Search\n", 404 | "Step: Searching the Web for: \"Macom recent news\"\n", 405 | "Step: Generating Final Response\n", 406 | "=======\n" 407 | ] 408 | }, 409 | { 410 | "data": { 411 | "text/markdown": [ 412 | "Based on the provided web search context, it appears that MACOM Technology Solutions Holdings, Inc. (MACOM) has been involved in several recent developments and announcements.\n", 413 | "\n", 414 | "In August 2023, MACOM announced that it had entered into a definitive agreement to acquire the radio frequency (RF) business of Wolfspeed, Inc. The acquisition was completed on December 2, 2023.\n", 415 | "\n", 416 | "In May 2024, MACOM announced its financial results for its fiscal second quarter ended March 29, 2024. The company reported revenue of $181.2 million, an increase of 7.0% compared to the previous year.\n", 417 | "\n", 418 | "Additionally, MACOM has been hosting live demonstrations of its products at industry events, including the Optical Fiber Communication Conference and Exhibition in March 2024.\n", 419 | "\n", 420 | "It's worth noting that while MACOM has experienced a recent dip in revenue growth (-8.95% over the last twelve months as of Q2 2024), the company has shown quarterly revenue growth of 6.98% in Q2 2024, outperforming the industry's decline of 0.4%.\n", 421 | "\n", 422 | "Overall, it appears that MACOM is focused on expanding its product portfolio and driving revenue growth through strategic acquisitions and product demonstrations." 423 | ], 424 | "text/plain": [ 425 | "" 426 | ] 427 | }, 428 | "metadata": {}, 429 | "output_type": "display_data" 430 | } 431 | ], 432 | "source": [ 433 | "# Test it out!\n", 434 | "run_agent(\"What's been up with Macom recently?\")" 435 | ] 436 | }, 437 | { 438 | "cell_type": "code", 439 | "execution_count": null, 440 | "id": "74c2eed6-c12e-4c8e-8e20-f8febbc0d0fb", 441 | "metadata": {}, 442 | "outputs": [], 443 | "source": [] 444 | } 445 | ], 446 | "metadata": { 447 | "kernelspec": { 448 | "display_name": "Python 3 (ipykernel)", 449 | "language": "python", 450 | "name": "python3" 451 | }, 452 | "language_info": { 453 | "codemirror_mode": { 454 | "name": "ipython", 455 | "version": 3 456 | }, 457 | "file_extension": ".py", 458 | "mimetype": "text/x-python", 459 | "name": "python", 460 | "nbconvert_exporter": "python", 461 | "pygments_lexer": "ipython3", 462 | "version": "3.12.1" 463 | } 464 | }, 465 | "nbformat": 4, 466 | "nbformat_minor": 5 467 | } 468 | --------------------------------------------------------------------------------