├── LICENSE ├── README.md ├── agents.ipynb ├── composability.ipynb ├── observability.ipynb ├── output_parsers.ipynb ├── requirements.txt ├── retrieval.ipynb └── streaming.ipynb /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Harrison Chase 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # langchain-0.1-guides 2 | 3 | These notebooks show off different functionality associated with `langchain` 0.1. 4 | 5 | 6 | ## Setup 7 | 8 | To setup, please install requirements: 9 | 10 | ```shell 11 | pip install -r requirements.txt 12 | ``` 13 | 14 | Then set required environment variables. 15 | We will use [OpenAI](https://platform.openai.com/docs/introduction) for our language model, and [Tavily](https://app.tavily.com/sign-in) for our search provider. 16 | 17 | ```shell 18 | export OPENAI_API_KEY=... 19 | export TAVILY_API_KEY=... 20 | ``` 21 | 22 | We will also use [LangSmith](https://docs.smith.langchain.com/) for observability: 23 | 24 | ```shell 25 | export LANGCHAIN_TRACING_V2="true" 26 | export LANGCHAIN_API_KEY=... 27 | ``` 28 | 29 | After that, we can start the Jupyter notebook server and follow along from there: 30 | 31 | ```shell 32 | jupyter notebook 33 | ``` 34 | 35 | ## Notebooks 36 | 37 | The following notebooks are provided: 38 | 39 | - [Observability](observability.ipynb) 40 | - [Composition](composability.ipynb) 41 | - [Streaming](streaming.ipynb) 42 | - [Output Parsing](output_parsers.ipynb) 43 | - [Retrieval](retrieval.ipynb) 44 | - [Agents](agents.ipynb) -------------------------------------------------------------------------------- /agents.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "78ec6921", 6 | "metadata": {}, 7 | "source": [ 8 | "# Agents\n", 9 | "\n", 10 | "This notebook covers basic functionality related to agents. For more in detail documentation, see:\n", 11 | "\n", 12 | "- [Agent Documentation](https://python.langchain.com/docs/modules/agents/)\n", 13 | "\n", 14 | "- [Agent Types](https://python.langchain.com/docs/modules/agents/agent_types/)" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 1, 20 | "id": "7b2421f4", 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "from langchain_community.tools.tavily_search import TavilySearchResults\n", 25 | "from langchain_openai import ChatOpenAI\n", 26 | "from langchain import hub\n", 27 | "from langchain.agents import create_openai_functions_agent\n", 28 | "from langchain.agents import AgentExecutor" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "id": "4d5efcf4", 34 | "metadata": {}, 35 | "source": [ 36 | "## Prompt" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": 2, 42 | "id": "5992682a", 43 | "metadata": {}, 44 | "outputs": [ 45 | { 46 | "name": "stderr", 47 | "output_type": "stream", 48 | "text": [ 49 | "/Users/harrisonchase/workplace/langchain/libs/core/langchain_core/_api/beta_decorator.py:160: LangChainBetaWarning: The function `loads` is in beta. It is actively being worked on, so the API may change.\n", 50 | " warn_beta(\n" 51 | ] 52 | } 53 | ], 54 | "source": [ 55 | "# Get the prompt to use - you can modify this!\n", 56 | "prompt = hub.pull(\"hwchase17/openai-functions-agent\")" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "id": "3716d768", 62 | "metadata": {}, 63 | "source": [ 64 | "## LLM" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": 3, 70 | "id": "30acf78d", 71 | "metadata": {}, 72 | "outputs": [], 73 | "source": [ 74 | "llm = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "id": "a3fd4cf9", 80 | "metadata": {}, 81 | "source": [ 82 | "## Tools" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": 4, 88 | "id": "32a2c52f", 89 | "metadata": {}, 90 | "outputs": [], 91 | "source": [ 92 | "search = TavilySearchResults()\n", 93 | "tools = [search]" 94 | ] 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "id": "7daee658", 99 | "metadata": {}, 100 | "source": [ 101 | "## Agent" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": 5, 107 | "id": "b6152ad4", 108 | "metadata": {}, 109 | "outputs": [], 110 | "source": [ 111 | "agent = create_openai_functions_agent(llm, tools, prompt)" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": 10, 117 | "id": "df123904", 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "result = agent.invoke({\"input\": \"what's the weather in SF?\", \"intermediate_steps\": []})" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": 11, 127 | "id": "fbf7ceb5", 128 | "metadata": {}, 129 | "outputs": [ 130 | { 131 | "data": { 132 | "text/plain": [ 133 | "'tavily_search_results_json'" 134 | ] 135 | }, 136 | "execution_count": 11, 137 | "metadata": {}, 138 | "output_type": "execute_result" 139 | } 140 | ], 141 | "source": [ 142 | "result.tool" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": 12, 148 | "id": "ee380cf3", 149 | "metadata": {}, 150 | "outputs": [ 151 | { 152 | "data": { 153 | "text/plain": [ 154 | "{'query': 'weather in San Francisco'}" 155 | ] 156 | }, 157 | "execution_count": 12, 158 | "metadata": {}, 159 | "output_type": "execute_result" 160 | } 161 | ], 162 | "source": [ 163 | "result.tool_input" 164 | ] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "execution_count": 9, 169 | "id": "55f14f30", 170 | "metadata": {}, 171 | "outputs": [ 172 | { 173 | "data": { 174 | "text/plain": [ 175 | "AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\\n \"query\": \"weather in San Francisco\"\\n}', 'name': 'tavily_search_results_json'}})])" 176 | ] 177 | }, 178 | "execution_count": 9, 179 | "metadata": {}, 180 | "output_type": "execute_result" 181 | } 182 | ], 183 | "source": [ 184 | "result" 185 | ] 186 | }, 187 | { 188 | "cell_type": "markdown", 189 | "id": "87a3993e", 190 | "metadata": {}, 191 | "source": [ 192 | "## Agent Executor" 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": 13, 198 | "id": "2eef3db8", 199 | "metadata": {}, 200 | "outputs": [], 201 | "source": [ 202 | "agent_executor = AgentExecutor(agent=agent, tools=tools)" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 14, 208 | "id": "a6c08822", 209 | "metadata": {}, 210 | "outputs": [ 211 | { 212 | "data": { 213 | "text/plain": [ 214 | "{'input': 'what is the weather in sf?',\n", 215 | " 'output': 'The weather in San Francisco is currently not available. However, you can check the weather in San Francisco in January 2024 [here](https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/).'}" 216 | ] 217 | }, 218 | "execution_count": 14, 219 | "metadata": {}, 220 | "output_type": "execute_result" 221 | } 222 | ], 223 | "source": [ 224 | "agent_executor.invoke({\"input\": \"what is the weather in sf?\"})" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "id": "4e8ca119", 230 | "metadata": {}, 231 | "source": [ 232 | "## Streaming" 233 | ] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": 15, 238 | "id": "e9b9dd6e", 239 | "metadata": {}, 240 | "outputs": [ 241 | { 242 | "name": "stdout", 243 | "output_type": "stream", 244 | "text": [ 245 | "{'actions': [AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\\n \"query\": \"weather in San Francisco\"\\n}', 'name': 'tavily_search_results_json'}})])], 'messages': [AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\\n \"query\": \"weather in San Francisco\"\\n}', 'name': 'tavily_search_results_json'}})]}\n", 246 | "{'steps': [AgentStep(action=AgentActionMessageLog(tool='tavily_search_results_json', tool_input={'query': 'weather in San Francisco'}, log=\"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\\n \"query\": \"weather in San Francisco\"\\n}', 'name': 'tavily_search_results_json'}})]), observation=[{'url': 'https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/', 'content': 'Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in january49°F Clear/Sunny 47% of time 24% 13% 8% Weather at 12pm 59°F Clear/Sunny 47% of time 26% 14% 9% Weather at 6pm 54°F Clear/Sunny 50% of time 23% 14%'}])], 'messages': [FunctionMessage(content='[{\"url\": \"https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/\", \"content\": \"Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in january49°F Clear/Sunny 47% of time 24% 13% 8% Weather at 12pm 59°F Clear/Sunny 47% of time 26% 14% 9% Weather at 6pm 54°F Clear/Sunny 50% of time 23% 14%\"}]', name='tavily_search_results_json')]}\n", 247 | "{'output': 'The weather in San Francisco is currently not available. However, you can check the weather in San Francisco in January 2024 [here](https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/).', 'messages': [AIMessage(content='The weather in San Francisco is currently not available. However, you can check the weather in San Francisco in January 2024 [here](https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/).')]}\n" 248 | ] 249 | } 250 | ], 251 | "source": [ 252 | "for step in agent_executor.stream({\"input\": \"what is the weather in sf?\"}):\n", 253 | " print(step)" 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": null, 259 | "id": "ba35f057", 260 | "metadata": {}, 261 | "outputs": [], 262 | "source": [] 263 | } 264 | ], 265 | "metadata": { 266 | "kernelspec": { 267 | "display_name": "Python 3 (ipykernel)", 268 | "language": "python", 269 | "name": "python3" 270 | }, 271 | "language_info": { 272 | "codemirror_mode": { 273 | "name": "ipython", 274 | "version": 3 275 | }, 276 | "file_extension": ".py", 277 | "mimetype": "text/x-python", 278 | "name": "python", 279 | "nbconvert_exporter": "python", 280 | "pygments_lexer": "ipython3", 281 | "version": "3.11.1" 282 | } 283 | }, 284 | "nbformat": 4, 285 | "nbformat_minor": 5 286 | } 287 | -------------------------------------------------------------------------------- /composability.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "a81ab12d-6512-4c27-8b82-9e12dec6d4d6", 6 | "metadata": {}, 7 | "source": [ 8 | "# Composability\n", 9 | "\n", 10 | "This notebook shows off some basic functionality around LangChain Expression Language, which makes it really easy to compose arbitrary chains.\n", 11 | "\n", 12 | "For a much deeper dive, see:\n", 13 | "\n", 14 | "- [Full LangChain Expression Language Documentation](https://python.langchain.com/docs/expression_language/)" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "id": "44b9f0ca", 20 | "metadata": {}, 21 | "source": [ 22 | "## Basic Composability" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 1, 28 | "id": "504f9999", 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "from langchain_openai import ChatOpenAI\n", 33 | "from langchain_core.prompts import ChatPromptTemplate\n", 34 | "from langchain_core.output_parsers import StrOutputParser" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": 2, 40 | "id": "c629ae08", 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "prompt = ChatPromptTemplate.from_template(\"Tell me a joke about {topic}\")\n", 45 | "model = ChatOpenAI()\n", 46 | "output_parser = StrOutputParser()" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": 3, 52 | "id": "47e25734", 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "chain = prompt | model | output_parser" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": 4, 62 | "id": "b97b0fed", 63 | "metadata": {}, 64 | "outputs": [ 65 | { 66 | "data": { 67 | "text/plain": [ 68 | "\"Why don't bears wear shoes? \\n\\nBecause they have bear feet!\"" 69 | ] 70 | }, 71 | "execution_count": 4, 72 | "metadata": {}, 73 | "output_type": "execute_result" 74 | } 75 | ], 76 | "source": [ 77 | "chain.invoke({\"topic\": \"bears\"})" 78 | ] 79 | }, 80 | { 81 | "cell_type": "markdown", 82 | "id": "1391ac95", 83 | "metadata": {}, 84 | "source": [ 85 | "### Batching" 86 | ] 87 | }, 88 | { 89 | "cell_type": "code", 90 | "execution_count": 5, 91 | "id": "edaab49c", 92 | "metadata": {}, 93 | "outputs": [ 94 | { 95 | "data": { 96 | "text/plain": [ 97 | "[\"Why don't bears like fast food?\\nBecause they can't catch it!\",\n", 98 | " \"Why don't clowns ever get angry? Because they always keep their cool!\"]" 99 | ] 100 | }, 101 | "execution_count": 5, 102 | "metadata": {}, 103 | "output_type": "execute_result" 104 | } 105 | ], 106 | "source": [ 107 | "chain.batch([{\"topic\": \"bear\"}, {\"topic\": \"clowns\"}])" 108 | ] 109 | }, 110 | { 111 | "cell_type": "markdown", 112 | "id": "2074e242", 113 | "metadata": {}, 114 | "source": [ 115 | "### Streaming" 116 | ] 117 | }, 118 | { 119 | "cell_type": "code", 120 | "execution_count": 6, 121 | "id": "5c29b910", 122 | "metadata": {}, 123 | "outputs": [ 124 | { 125 | "name": "stdout", 126 | "output_type": "stream", 127 | "text": [ 128 | "\n", 129 | "Why\n", 130 | " don\n", 131 | "'t\n", 132 | " bears\n", 133 | " wear\n", 134 | " shoes\n", 135 | "?\n", 136 | "\n", 137 | "\n", 138 | "Because\n", 139 | " they\n", 140 | " have\n", 141 | " bear\n", 142 | " feet\n", 143 | "!\n", 144 | "\n" 145 | ] 146 | } 147 | ], 148 | "source": [ 149 | "for s in chain.stream({\"topic\": \"bears\"}):\n", 150 | " print(s)" 151 | ] 152 | }, 153 | { 154 | "cell_type": "markdown", 155 | "id": "caa5be86", 156 | "metadata": {}, 157 | "source": [ 158 | "## RunnablePassthrough" 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": 7, 164 | "id": "e0ed6b75", 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "from langchain_community.retrievers.tavily_search_api import TavilySearchAPIRetriever\n", 169 | "\n", 170 | "retriever= TavilySearchAPIRetriever()\n", 171 | "\n", 172 | "prompt = ChatPromptTemplate.from_template(\"\"\"Answer the question based only on the context provided:\n", 173 | "\n", 174 | "Context: {context}\n", 175 | "\n", 176 | "Question: {question}\"\"\")" 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": 8, 182 | "id": "94e75648", 183 | "metadata": {}, 184 | "outputs": [], 185 | "source": [ 186 | "chain = prompt | model | output_parser" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": 9, 192 | "id": "abcbf78f", 193 | "metadata": {}, 194 | "outputs": [ 195 | { 196 | "data": { 197 | "text/plain": [ 198 | "'Langsmith is a testing and observability platform developed by the Langchain team.'" 199 | ] 200 | }, 201 | "execution_count": 9, 202 | "metadata": {}, 203 | "output_type": "execute_result" 204 | } 205 | ], 206 | "source": [ 207 | "question = \"what is langsmith\"\n", 208 | "context = \"langsmith is a testing and observability platform built by the langchain team\"\n", 209 | "chain.invoke({\"question\": question, \"context\": context})" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": 10, 215 | "id": "98b8a905", 216 | "metadata": {}, 217 | "outputs": [], 218 | "source": [ 219 | "from langchain_core.runnables import RunnablePassthrough\n", 220 | "\n", 221 | "retrieval_chain = RunnablePassthrough.assign(\n", 222 | " context=(lambda x: x[\"question\"]) | retriever\n", 223 | ") | chain" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": 11, 229 | "id": "4c488db8", 230 | "metadata": {}, 231 | "outputs": [ 232 | { 233 | "data": { 234 | "text/plain": [ 235 | "'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any Language Model (LLM) framework. LangSmith seamlessly integrates with LangChain, an open source framework for building with LLMs.'" 236 | ] 237 | }, 238 | "execution_count": 11, 239 | "metadata": {}, 240 | "output_type": "execute_result" 241 | } 242 | ], 243 | "source": [ 244 | "retrieval_chain.invoke({\"question\": \"what is langsmith\"})" 245 | ] 246 | }, 247 | { 248 | "cell_type": "markdown", 249 | "id": "7f5cff2b", 250 | "metadata": {}, 251 | "source": [ 252 | "## RunnableParallel" 253 | ] 254 | }, 255 | { 256 | "cell_type": "code", 257 | "execution_count": 12, 258 | "id": "b36365f0", 259 | "metadata": {}, 260 | "outputs": [], 261 | "source": [ 262 | "from langchain_core.runnables import RunnableParallel" 263 | ] 264 | }, 265 | { 266 | "cell_type": "code", 267 | "execution_count": 13, 268 | "id": "2d2ca55c", 269 | "metadata": {}, 270 | "outputs": [], 271 | "source": [ 272 | "prompt = ChatPromptTemplate.from_template(\"\"\"{question}\"\"\")\n", 273 | "simple_chain = prompt | model | output_parser" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": 14, 279 | "id": "b2e2db81", 280 | "metadata": {}, 281 | "outputs": [], 282 | "source": [ 283 | "parallel_chain = RunnableParallel({\n", 284 | " \"retrieved_answer\": retrieval_chain,\n", 285 | " \"simple_answer\": simple_chain\n", 286 | "})" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": 15, 292 | "id": "21db1151", 293 | "metadata": {}, 294 | "outputs": [ 295 | { 296 | "data": { 297 | "text/plain": [ 298 | "{'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any Language Model (LLM) framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs.',\n", 299 | " 'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It does not appear to be a widely known term or concept.'}" 300 | ] 301 | }, 302 | "execution_count": 15, 303 | "metadata": {}, 304 | "output_type": "execute_result" 305 | } 306 | ], 307 | "source": [ 308 | "parallel_chain.invoke({\"question\": \"what is langsmith\"})" 309 | ] 310 | }, 311 | { 312 | "cell_type": "code", 313 | "execution_count": 16, 314 | "id": "39a00772", 315 | "metadata": {}, 316 | "outputs": [ 317 | { 318 | "name": "stdout", 319 | "output_type": "stream", 320 | "text": [ 321 | "{'simple_answer': ''}\n", 322 | "{'simple_answer': 'I'}\n", 323 | "{'simple_answer': \"'m\"}\n", 324 | "{'simple_answer': ' sorry'}\n", 325 | "{'simple_answer': ','}\n", 326 | "{'simple_answer': ' but'}\n", 327 | "{'simple_answer': ' I'}\n", 328 | "{'simple_answer': ' couldn'}\n", 329 | "{'simple_answer': \"'t\"}\n", 330 | "{'simple_answer': ' find'}\n", 331 | "{'simple_answer': ' any'}\n", 332 | "{'simple_answer': ' information'}\n", 333 | "{'simple_answer': ' on'}\n", 334 | "{'simple_answer': ' \"'}\n", 335 | "{'simple_answer': 'lang'}\n", 336 | "{'simple_answer': 'smith'}\n", 337 | "{'simple_answer': '.\"'}\n", 338 | "{'simple_answer': ' It'}\n", 339 | "{'simple_answer': ' is'}\n", 340 | "{'simple_answer': ' possible'}\n", 341 | "{'simple_answer': ' that'}\n", 342 | "{'simple_answer': ' it'}\n", 343 | "{'simple_answer': ' may'}\n", 344 | "{'simple_answer': ' be'}\n", 345 | "{'simple_answer': ' a'}\n", 346 | "{'simple_answer': ' lesser'}\n", 347 | "{'simple_answer': '-known'}\n", 348 | "{'simple_answer': ' term'}\n", 349 | "{'simple_answer': ' or'}\n", 350 | "{'simple_answer': ' a'}\n", 351 | "{'simple_answer': ' specific'}\n", 352 | "{'simple_answer': ' reference'}\n", 353 | "{'simple_answer': ' that'}\n", 354 | "{'simple_answer': ' requires'}\n", 355 | "{'simple_answer': ' more'}\n", 356 | "{'simple_answer': ' context'}\n", 357 | "{'simple_answer': '.'}\n", 358 | "{'simple_answer': ' Could'}\n", 359 | "{'simple_answer': ' you'}\n", 360 | "{'simple_answer': ' please'}\n", 361 | "{'simple_answer': ' provide'}\n", 362 | "{'simple_answer': ' more'}\n", 363 | "{'simple_answer': ' information'}\n", 364 | "{'simple_answer': ' or'}\n", 365 | "{'simple_answer': ' clarify'}\n", 366 | "{'simple_answer': ' your'}\n", 367 | "{'simple_answer': ' question'}\n", 368 | "{'simple_answer': '?'}\n", 369 | "{'simple_answer': ''}\n", 370 | "{'retrieved_answer': ''}\n", 371 | "{'retrieved_answer': 'Lang'}\n", 372 | "{'retrieved_answer': 'Smith'}\n", 373 | "{'retrieved_answer': ' is'}\n", 374 | "{'retrieved_answer': ' a'}\n", 375 | "{'retrieved_answer': ' platform'}\n", 376 | "{'retrieved_answer': ' that'}\n", 377 | "{'retrieved_answer': ' helps'}\n", 378 | "{'retrieved_answer': ' trace'}\n", 379 | "{'retrieved_answer': ','}\n", 380 | "{'retrieved_answer': ' evaluate'}\n", 381 | "{'retrieved_answer': ','}\n", 382 | "{'retrieved_answer': ' debug'}\n", 383 | "{'retrieved_answer': ','}\n", 384 | "{'retrieved_answer': ' test'}\n", 385 | "{'retrieved_answer': ','}\n", 386 | "{'retrieved_answer': ' and'}\n", 387 | "{'retrieved_answer': ' monitor'}\n", 388 | "{'retrieved_answer': ' language'}\n", 389 | "{'retrieved_answer': ' model'}\n", 390 | "{'retrieved_answer': ' applications'}\n", 391 | "{'retrieved_answer': ' and'}\n", 392 | "{'retrieved_answer': ' intelligent'}\n", 393 | "{'retrieved_answer': ' agents'}\n", 394 | "{'retrieved_answer': ' built'}\n", 395 | "{'retrieved_answer': ' on'}\n", 396 | "{'retrieved_answer': ' any'}\n", 397 | "{'retrieved_answer': ' Language'}\n", 398 | "{'retrieved_answer': ' Model'}\n", 399 | "{'retrieved_answer': ' ('}\n", 400 | "{'retrieved_answer': 'LL'}\n", 401 | "{'retrieved_answer': 'M'}\n", 402 | "{'retrieved_answer': ')'}\n", 403 | "{'retrieved_answer': ' framework'}\n", 404 | "{'retrieved_answer': '.'}\n", 405 | "{'retrieved_answer': ' It'}\n", 406 | "{'retrieved_answer': ' is'}\n", 407 | "{'retrieved_answer': ' a'}\n", 408 | "{'retrieved_answer': ' unified'}\n", 409 | "{'retrieved_answer': ' platform'}\n", 410 | "{'retrieved_answer': ' that'}\n", 411 | "{'retrieved_answer': ' provides'}\n", 412 | "{'retrieved_answer': ' visibility'}\n", 413 | "{'retrieved_answer': ' into'}\n", 414 | "{'retrieved_answer': ' model'}\n", 415 | "{'retrieved_answer': ' inputs'}\n", 416 | "{'retrieved_answer': ' and'}\n", 417 | "{'retrieved_answer': ' outputs'}\n", 418 | "{'retrieved_answer': ','}\n", 419 | "{'retrieved_answer': ' allows'}\n", 420 | "{'retrieved_answer': ' for'}\n", 421 | "{'retrieved_answer': ' dataset'}\n", 422 | "{'retrieved_answer': ' creation'}\n", 423 | "{'retrieved_answer': ' and'}\n", 424 | "{'retrieved_answer': ' testing'}\n", 425 | "{'retrieved_answer': ','}\n", 426 | "{'retrieved_answer': ' and'}\n", 427 | "{'retrieved_answer': ' offers'}\n", 428 | "{'retrieved_answer': ' tools'}\n", 429 | "{'retrieved_answer': ' for'}\n", 430 | "{'retrieved_answer': ' fine'}\n", 431 | "{'retrieved_answer': '-t'}\n", 432 | "{'retrieved_answer': 'uning'}\n", 433 | "{'retrieved_answer': ' and'}\n", 434 | "{'retrieved_answer': ' exporting'}\n", 435 | "{'retrieved_answer': ' data'}\n", 436 | "{'retrieved_answer': ' for'}\n", 437 | "{'retrieved_answer': ' analysis'}\n", 438 | "{'retrieved_answer': '.'}\n", 439 | "{'retrieved_answer': ' Lang'}\n", 440 | "{'retrieved_answer': 'Smith'}\n", 441 | "{'retrieved_answer': ' seamlessly'}\n", 442 | "{'retrieved_answer': ' integrates'}\n", 443 | "{'retrieved_answer': ' with'}\n", 444 | "{'retrieved_answer': ' Lang'}\n", 445 | "{'retrieved_answer': 'Chain'}\n", 446 | "{'retrieved_answer': ','}\n", 447 | "{'retrieved_answer': ' an'}\n", 448 | "{'retrieved_answer': ' open'}\n", 449 | "{'retrieved_answer': '-source'}\n", 450 | "{'retrieved_answer': ' framework'}\n", 451 | "{'retrieved_answer': ' for'}\n", 452 | "{'retrieved_answer': ' building'}\n", 453 | "{'retrieved_answer': ' with'}\n", 454 | "{'retrieved_answer': ' L'}\n", 455 | "{'retrieved_answer': 'LM'}\n", 456 | "{'retrieved_answer': 's'}\n", 457 | "{'retrieved_answer': '.'}\n", 458 | "{'retrieved_answer': ''}\n" 459 | ] 460 | } 461 | ], 462 | "source": [ 463 | "for s in parallel_chain.stream({\"question\": \"what is langsmith\"}):\n", 464 | " print(s)" 465 | ] 466 | }, 467 | { 468 | "cell_type": "code", 469 | "execution_count": 17, 470 | "id": "5a749099", 471 | "metadata": {}, 472 | "outputs": [ 473 | { 474 | "name": "stdout", 475 | "output_type": "stream", 476 | "text": [ 477 | "{'simple_answer': ''}\n", 478 | "{'simple_answer': 'I'}\n", 479 | "{'simple_answer': \"I'm\"}\n", 480 | "{'simple_answer': \"I'm sorry\"}\n", 481 | "{'simple_answer': \"I'm sorry,\"}\n", 482 | "{'simple_answer': \"I'm sorry, but\"}\n", 483 | "{'simple_answer': \"I'm sorry, but I\"}\n", 484 | "{'simple_answer': \"I'm sorry, but I couldn\"}\n", 485 | "{'simple_answer': \"I'm sorry, but I couldn't\"}\n", 486 | "{'simple_answer': \"I'm sorry, but I couldn't find\"}\n", 487 | "{'simple_answer': \"I'm sorry, but I couldn't find any\"}\n", 488 | "{'simple_answer': \"I'm sorry, but I couldn't find any information\"}\n", 489 | "{'simple_answer': \"I'm sorry, but I couldn't find any information on\"}\n", 490 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"'}\n", 491 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Lang'}\n", 492 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith'}\n", 493 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\"'}\n", 494 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It'}\n", 495 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might'}\n", 496 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be'}\n", 497 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a'}\n", 498 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a miss'}\n", 499 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a missp'}\n", 500 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling'}\n", 501 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or'}\n", 502 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a'}\n", 503 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less'}\n", 504 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known'}\n", 505 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term'}\n", 506 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term.'}\n", 507 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could'}\n", 508 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you'}\n", 509 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please'}\n", 510 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide'}\n", 511 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more'}\n", 512 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context'}\n", 513 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or'}\n", 514 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify'}\n", 515 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your'}\n", 516 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question'}\n", 517 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?'}\n", 518 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?'}\n", 519 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': ''}\n", 520 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'Lang'}\n", 521 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith'}\n", 522 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is'}\n", 523 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a'}\n", 524 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform'}\n", 525 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that'}\n", 526 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps'}\n", 527 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace'}\n", 528 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and'}\n", 529 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate'}\n", 530 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language'}\n", 531 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model'}\n", 532 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications'}\n", 533 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and'}\n", 534 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent'}\n", 535 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents'}\n", 536 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents.'}\n", 537 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It'}\n", 538 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows'}\n", 539 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users'}\n", 540 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to'}\n", 541 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug'}\n", 542 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug,'}\n", 543 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test'}\n", 544 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test,'}\n", 545 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate'}\n", 546 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate,'}\n", 547 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and'}\n", 548 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor'}\n", 549 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains'}\n", 550 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and'}\n", 551 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent'}\n", 552 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents'}\n", 553 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built'}\n", 554 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on'}\n", 555 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any'}\n", 556 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any L'}\n", 557 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM'}\n", 558 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework'}\n", 559 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework.'}\n", 560 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. Lang'}\n", 561 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith'}\n", 562 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly'}\n", 563 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates'}\n", 564 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with'}\n", 565 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with Lang'}\n", 566 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain'}\n", 567 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain,'}\n", 568 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an'}\n", 569 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open'}\n", 570 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source'}\n", 571 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework'}\n", 572 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for'}\n", 573 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building'}\n", 574 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with'}\n", 575 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with L'}\n", 576 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLM'}\n", 577 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs'}\n", 578 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs.'}\n", 579 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It'}\n", 580 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also'}\n", 581 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides'}\n", 582 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a'}\n", 583 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified'}\n", 584 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform'}\n", 585 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for'}\n", 586 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking'}\n", 587 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system'}\n", 588 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level'}\n", 589 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and'}\n", 590 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model'}\n", 591 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/'}\n", 592 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain'}\n", 593 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance'}\n", 594 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance,'}\n", 595 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging'}\n", 596 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues'}\n", 597 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues,'}\n", 598 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and'}\n", 599 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and understanding'}\n", 600 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and understanding user'}\n", 601 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and understanding user interactions'}\n", 602 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and understanding user interactions with'}\n", 603 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and understanding user interactions with applications'}\n", 604 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and understanding user interactions with applications.'}\n", 605 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"Langsmith.\" It might be a misspelling or a less-known term. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that helps trace and evaluate language model applications and intelligent agents. It allows users to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain, an open-source framework for building with LLMs. It also provides a unified platform for tracking system-level and model/chain performance, debugging issues, and understanding user interactions with applications.'}\n" 606 | ] 607 | }, 608 | { 609 | "name": "stdout", 610 | "output_type": "stream", 611 | "text": [ 612 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility'}\n", 613 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into'}\n", 614 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model'}\n", 615 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs'}\n", 616 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and'}\n", 617 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs'}\n", 618 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs,'}\n", 619 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and'}\n", 620 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing'}\n", 621 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for'}\n", 622 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the'}\n", 623 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement'}\n", 624 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement,'}\n", 625 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing'}\n", 626 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing,'}\n", 627 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and'}\n", 628 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and version'}\n", 629 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning'}\n", 630 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of'}\n", 631 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts'}\n", 632 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts.'}\n", 633 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. Lang'}\n", 634 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith'}\n", 635 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also'}\n", 636 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides'}\n", 637 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools'}\n", 638 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for'}\n", 639 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset'}\n", 640 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset c'}\n", 641 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation'}\n", 642 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation,'}\n", 643 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance'}\n", 644 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance monitoring'}\n", 645 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance monitoring,'}\n", 646 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance monitoring, and'}\n", 647 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance monitoring, and feedback'}\n", 648 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance monitoring, and feedback collection'}\n", 649 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance monitoring, and feedback collection.'}\n", 650 | "{'simple_answer': 'I\\'m sorry, but I couldn\\'t find any information on \"langsmith.\" It seems to be a term that is not widely recognized or used. Could you please provide more context or clarify your question?', 'retrieved_answer': 'LangSmith is a platform that offers features for debugging, testing, evaluating, and monitoring Language Learning Models (LLMs) and AI applications. It helps users trace and evaluate their language model applications and intelligent agents, providing visibility into model inputs and outputs, and allowing for the refinement, testing, and versioning of prompts. LangSmith also provides tools for dataset curation, performance monitoring, and feedback collection.'}\n" 651 | ] 652 | } 653 | ], 654 | "source": [ 655 | "result = {}\n", 656 | "for s in parallel_chain.stream({\"question\": \"what is langsmith\"}):\n", 657 | " for k,v in s.items():\n", 658 | " if k not in result:\n", 659 | " result[k] = \"\"\n", 660 | " result[k] += v\n", 661 | " print(result)" 662 | ] 663 | }, 664 | { 665 | "cell_type": "code", 666 | "execution_count": null, 667 | "id": "5d6b5174", 668 | "metadata": {}, 669 | "outputs": [], 670 | "source": [] 671 | } 672 | ], 673 | "metadata": { 674 | "kernelspec": { 675 | "display_name": "Python 3 (ipykernel)", 676 | "language": "python", 677 | "name": "python3" 678 | }, 679 | "language_info": { 680 | "codemirror_mode": { 681 | "name": "ipython", 682 | "version": 3 683 | }, 684 | "file_extension": ".py", 685 | "mimetype": "text/x-python", 686 | "name": "python", 687 | "nbconvert_exporter": "python", 688 | "pygments_lexer": "ipython3", 689 | "version": "3.11.1" 690 | } 691 | }, 692 | "nbformat": 4, 693 | "nbformat_minor": 5 694 | } 695 | -------------------------------------------------------------------------------- /observability.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "cbe8c769-b127-4a24-8c9f-aa8e979bfeb1", 6 | "metadata": {}, 7 | "source": [ 8 | "# Observability\n", 9 | "\n", 10 | "This notebook shows off different levels of LangChain observability. For more information, see:\n", 11 | "\n", 12 | "- [LangChain Debugging Guide](https://python.langchain.com/docs/guides/debugging)\n", 13 | "\n", 14 | "- [LangSmith](https://smith.langchain.com/)" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "id": "065a5d5b", 20 | "metadata": {}, 21 | "source": [ 22 | "## No Logging" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 1, 28 | "id": "7fd2faed", 29 | "metadata": {}, 30 | "outputs": [ 31 | { 32 | "name": "stderr", 33 | "output_type": "stream", 34 | "text": [ 35 | "/Users/harrisonchase/.pyenv/versions/3.11.1/envs/langchain-0.1-guides/lib/python3.11/site-packages/langchain_core/_api/beta_decorator.py:160: LangChainBetaWarning: The function `loads` is in beta. It is actively being worked on, so the API may change.\n", 36 | " warn_beta(\n" 37 | ] 38 | } 39 | ], 40 | "source": [ 41 | "from langchain_community.tools.tavily_search import TavilySearchResults\n", 42 | "from langchain_openai import ChatOpenAI\n", 43 | "from langchain import hub\n", 44 | "from langchain.agents import create_openai_functions_agent\n", 45 | "from langchain.agents import AgentExecutor\n", 46 | "\n", 47 | "# Get the prompt to use - you can modify this!\n", 48 | "prompt = hub.pull(\"hwchase17/openai-functions-agent\")\n", 49 | "llm = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)\n", 50 | "search = TavilySearchResults()\n", 51 | "tools = [search]\n", 52 | "agent = create_openai_functions_agent(llm, tools, prompt)\n", 53 | "agent_executor = AgentExecutor(agent=agent, tools=tools)" 54 | ] 55 | }, 56 | { 57 | "cell_type": "code", 58 | "execution_count": 2, 59 | "id": "eed69cda", 60 | "metadata": {}, 61 | "outputs": [ 62 | { 63 | "data": { 64 | "text/plain": [ 65 | "{'input': 'what is the weather in sf?',\n", 66 | " 'output': \"I found some information about the weather in San Francisco. According to the search results, the weather in San Francisco in January 2024 is expected to have an average temperature ranging from 52°F to 58°F. However, I couldn't find the current weather conditions. It's always a good idea to check a reliable weather website or app for the most up-to-date information.\"}" 67 | ] 68 | }, 69 | "execution_count": 2, 70 | "metadata": {}, 71 | "output_type": "execute_result" 72 | } 73 | ], 74 | "source": [ 75 | "agent_executor.invoke({\"input\": \"what is the weather in sf?\"})" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "id": "e9ee9b28", 81 | "metadata": {}, 82 | "source": [ 83 | "## `verbose=True`" 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": 3, 89 | "id": "c47044da", 90 | "metadata": {}, 91 | "outputs": [ 92 | { 93 | "name": "stderr", 94 | "output_type": "stream", 95 | "text": [ 96 | "/Users/harrisonchase/.pyenv/versions/3.11.1/envs/langchain-0.1-guides/lib/python3.11/site-packages/langchain_core/_api/beta_decorator.py:160: LangChainBetaWarning: The function `loads` is in beta. It is actively being worked on, so the API may change.\n", 97 | " warn_beta(\n" 98 | ] 99 | } 100 | ], 101 | "source": [ 102 | "from langchain.globals import set_verbose\n", 103 | "\n", 104 | "set_verbose(True)\n", 105 | "\n", 106 | "prompt = hub.pull(\"hwchase17/openai-functions-agent\")\n", 107 | "llm = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)\n", 108 | "search = TavilySearchResults()\n", 109 | "tools = [search]\n", 110 | "agent = create_openai_functions_agent(llm, tools, prompt)\n", 111 | "agent_executor = AgentExecutor(agent=agent, tools=tools)" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": 4, 117 | "id": "e0377506", 118 | "metadata": {}, 119 | "outputs": [ 120 | { 121 | "name": "stdout", 122 | "output_type": "stream", 123 | "text": [ 124 | "\n", 125 | "\n", 126 | "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", 127 | "\u001b[32;1m\u001b[1;3m\n", 128 | "Invoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\n", 129 | "\n", 130 | "\n", 131 | "\u001b[0m\u001b[36;1m\u001b[1;3m[{'url': 'https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/', 'content': 'Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in januaryWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 07-01-2023 54°F to 56°F. 08-01-2023 52°F to 58°F. 09-01-2023 54°F to ...'}, {'url': 'https://www.oddsshopper.com/articles/nfl/los-angeles-rams-san-francisco-49ers-weather-report-01-07-2024', 'content': \"find a current San Francisco-Los Angeles weather report. Weather Forecasted for the Rams-49ers Football Game November 9, 2023 7:58 am 7:58 am Rams-49ers Weather ReportJanuary 07, 2024 | 2:03 AM by OddsShopper Staff. This Week 18 NFL game is to be played in Santa Clara, CA at Levi's Stadium. Below you can find a current San Francisco-Los Angeles weather report. 🏈 Game time: 4:25PM EST: 📅 Game date: Sunday, January 7th, 2024: ... 01 Bears vs Packers 22:41 Chiefs vs Chargers 25:08 Broncos vs Raiders\"}, {'url': 'https://www.mercurynews.com/2024/01/07/bay-area-and-beyond-frost-advisory-and-freeze-warning-monday-morning/', 'content': 'degrees Fahrenheit respectively. In San Francisco, meanwhile, the low is forecasted to remain a relatively balmy 41 Weather | Bay Area will approach freezing temps overnight with light rain and large waves Weather | Weather | Bay Area temps could dip to freezing overnight after rain, heavy snowfall pound the Sierra Nevada Weather | Weather | 7 days of rain: How much did your city get during new year’s storms? Copyright © 2024 MediaNews GroupJanuary 7, 2024 at 10:49 a.m. After an unseasonably warm December, the Bay Area is ringing in the new year with freezing temperatures. According to the National Weather Service, interior parts of ...'}]\u001b[0m\u001b[32;1m\u001b[1;3mI found some information about the weather in San Francisco. According to the search results, the weather in San Francisco in January 2024 is expected to have an average temperature ranging from 52°F to 58°F. However, I couldn't find the current weather conditions. It's always a good idea to check a reliable weather website or app for the most up-to-date information.\u001b[0m\n", 132 | "\n", 133 | "\u001b[1m> Finished chain.\u001b[0m\n" 134 | ] 135 | }, 136 | { 137 | "data": { 138 | "text/plain": [ 139 | "{'input': 'what is the weather in sf?',\n", 140 | " 'output': \"I found some information about the weather in San Francisco. According to the search results, the weather in San Francisco in January 2024 is expected to have an average temperature ranging from 52°F to 58°F. However, I couldn't find the current weather conditions. It's always a good idea to check a reliable weather website or app for the most up-to-date information.\"}" 141 | ] 142 | }, 143 | "execution_count": 4, 144 | "metadata": {}, 145 | "output_type": "execute_result" 146 | } 147 | ], 148 | "source": [ 149 | "agent_executor.invoke({\"input\": \"what is the weather in sf?\"})" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "id": "0014dd98", 155 | "metadata": {}, 156 | "source": [ 157 | "## `debug=True`" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": 5, 163 | "id": "7f2295f9", 164 | "metadata": {}, 165 | "outputs": [ 166 | { 167 | "name": "stderr", 168 | "output_type": "stream", 169 | "text": [ 170 | "/Users/harrisonchase/.pyenv/versions/3.11.1/envs/langchain-0.1-guides/lib/python3.11/site-packages/langchain_core/_api/beta_decorator.py:160: LangChainBetaWarning: The function `loads` is in beta. It is actively being worked on, so the API may change.\n", 171 | " warn_beta(\n" 172 | ] 173 | } 174 | ], 175 | "source": [ 176 | "from langchain.globals import set_debug\n", 177 | "\n", 178 | "set_verbose(False)\n", 179 | "set_debug(True)\n", 180 | "\n", 181 | "prompt = hub.pull(\"hwchase17/openai-functions-agent\")\n", 182 | "llm = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)\n", 183 | "search = TavilySearchResults()\n", 184 | "tools = [search]\n", 185 | "agent = create_openai_functions_agent(llm, tools, prompt)\n", 186 | "agent_executor = AgentExecutor(agent=agent, tools=tools)" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": 6, 192 | "id": "66acc1e1", 193 | "metadata": {}, 194 | "outputs": [ 195 | { 196 | "name": "stdout", 197 | "output_type": "stream", 198 | "text": [ 199 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor] Entering Chain run with input:\n", 200 | "\u001b[0m{\n", 201 | " \"input\": \"what is the weather in sf?\"\n", 202 | "}\n", 203 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence] Entering Chain run with input:\n", 204 | "\u001b[0m{\n", 205 | " \"input\": \"what is the weather in sf?\",\n", 206 | " \"intermediate_steps\": []\n", 207 | "}\n", 208 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 3:chain:RunnableAssign] Entering Chain run with input:\n", 209 | "\u001b[0m{\n", 210 | " \"input\": \"what is the weather in sf?\",\n", 211 | " \"intermediate_steps\": []\n", 212 | "}\n", 213 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 3:chain:RunnableAssign > 4:chain:RunnableParallel] Entering Chain run with input:\n", 214 | "\u001b[0m{\n", 215 | " \"input\": \"what is the weather in sf?\",\n", 216 | " \"intermediate_steps\": []\n", 217 | "}\n", 218 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 3:chain:RunnableAssign > 4:chain:RunnableParallel > 5:chain:] Entering Chain run with input:\n", 219 | "\u001b[0m{\n", 220 | " \"input\": \"what is the weather in sf?\",\n", 221 | " \"intermediate_steps\": []\n", 222 | "}\n", 223 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 3:chain:RunnableAssign > 4:chain:RunnableParallel > 5:chain:] [0ms] Exiting Chain run with output:\n", 224 | "\u001b[0m{\n", 225 | " \"output\": []\n", 226 | "}\n", 227 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 3:chain:RunnableAssign > 4:chain:RunnableParallel] [1ms] Exiting Chain run with output:\n", 228 | "\u001b[0m{\n", 229 | " \"agent_scratchpad\": []\n", 230 | "}\n", 231 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 3:chain:RunnableAssign] [1ms] Exiting Chain run with output:\n", 232 | "\u001b[0m{\n", 233 | " \"input\": \"what is the weather in sf?\",\n", 234 | " \"intermediate_steps\": [],\n", 235 | " \"agent_scratchpad\": []\n", 236 | "}\n", 237 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 6:prompt:ChatPromptTemplate] Entering Prompt run with input:\n", 238 | "\u001b[0m{\n", 239 | " \"input\": \"what is the weather in sf?\",\n", 240 | " \"intermediate_steps\": [],\n", 241 | " \"agent_scratchpad\": []\n", 242 | "}\n", 243 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 6:prompt:ChatPromptTemplate] [0ms] Exiting Prompt run with output:\n", 244 | "\u001b[0m{\n", 245 | " \"lc\": 1,\n", 246 | " \"type\": \"constructor\",\n", 247 | " \"id\": [\n", 248 | " \"langchain\",\n", 249 | " \"prompts\",\n", 250 | " \"chat\",\n", 251 | " \"ChatPromptValue\"\n", 252 | " ],\n", 253 | " \"kwargs\": {\n", 254 | " \"messages\": [\n", 255 | " {\n", 256 | " \"lc\": 1,\n", 257 | " \"type\": \"constructor\",\n", 258 | " \"id\": [\n", 259 | " \"langchain\",\n", 260 | " \"schema\",\n", 261 | " \"messages\",\n", 262 | " \"SystemMessage\"\n", 263 | " ],\n", 264 | " \"kwargs\": {\n", 265 | " \"content\": \"You are a helpful assistant\",\n", 266 | " \"additional_kwargs\": {}\n", 267 | " }\n", 268 | " },\n", 269 | " {\n", 270 | " \"lc\": 1,\n", 271 | " \"type\": \"constructor\",\n", 272 | " \"id\": [\n", 273 | " \"langchain\",\n", 274 | " \"schema\",\n", 275 | " \"messages\",\n", 276 | " \"HumanMessage\"\n", 277 | " ],\n", 278 | " \"kwargs\": {\n", 279 | " \"content\": \"what is the weather in sf?\",\n", 280 | " \"additional_kwargs\": {}\n", 281 | " }\n", 282 | " }\n", 283 | " ]\n", 284 | " }\n", 285 | "}\n", 286 | "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 7:llm:ChatOpenAI] Entering LLM run with input:\n", 287 | "\u001b[0m{\n", 288 | " \"prompts\": [\n", 289 | " \"System: You are a helpful assistant\\nHuman: what is the weather in sf?\"\n", 290 | " ]\n", 291 | "}\n", 292 | "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 7:llm:ChatOpenAI] [546ms] Exiting LLM run with output:\n", 293 | "\u001b[0m{\n", 294 | " \"generations\": [\n", 295 | " [\n", 296 | " {\n", 297 | " \"text\": \"\",\n", 298 | " \"generation_info\": {\n", 299 | " \"finish_reason\": \"function_call\",\n", 300 | " \"logprobs\": null\n", 301 | " },\n", 302 | " \"type\": \"ChatGeneration\",\n", 303 | " \"message\": {\n", 304 | " \"lc\": 1,\n", 305 | " \"type\": \"constructor\",\n", 306 | " \"id\": [\n", 307 | " \"langchain\",\n", 308 | " \"schema\",\n", 309 | " \"messages\",\n", 310 | " \"AIMessage\"\n", 311 | " ],\n", 312 | " \"kwargs\": {\n", 313 | " \"content\": \"\",\n", 314 | " \"additional_kwargs\": {\n", 315 | " \"function_call\": {\n", 316 | " \"arguments\": \"{\\n \\\"query\\\": \\\"weather in San Francisco\\\"\\n}\",\n", 317 | " \"name\": \"tavily_search_results_json\"\n", 318 | " }\n", 319 | " }\n", 320 | " }\n", 321 | " }\n", 322 | " }\n", 323 | " ]\n", 324 | " ],\n", 325 | " \"llm_output\": {\n", 326 | " \"token_usage\": {\n", 327 | " \"completion_tokens\": 22,\n", 328 | " \"prompt_tokens\": 94,\n", 329 | " \"total_tokens\": 116\n", 330 | " },\n", 331 | " \"model_name\": \"gpt-3.5-turbo\",\n", 332 | " \"system_fingerprint\": null\n", 333 | " },\n", 334 | " \"run\": null\n", 335 | "}\n", 336 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 8:parser:OpenAIFunctionsAgentOutputParser] Entering Parser run with input:\n", 337 | "\u001b[0m[inputs]\n", 338 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence > 8:parser:OpenAIFunctionsAgentOutputParser] [1ms] Exiting Parser run with output:\n", 339 | "\u001b[0m{\n", 340 | " \"lc\": 1,\n", 341 | " \"type\": \"constructor\",\n", 342 | " \"id\": [\n", 343 | " \"langchain\",\n", 344 | " \"schema\",\n", 345 | " \"agent\",\n", 346 | " \"AgentActionMessageLog\"\n", 347 | " ],\n", 348 | " \"kwargs\": {\n", 349 | " \"tool\": \"tavily_search_results_json\",\n", 350 | " \"tool_input\": {\n", 351 | " \"query\": \"weather in San Francisco\"\n", 352 | " },\n", 353 | " \"log\": \"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\",\n", 354 | " \"message_log\": [\n", 355 | " {\n", 356 | " \"lc\": 1,\n", 357 | " \"type\": \"constructor\",\n", 358 | " \"id\": [\n", 359 | " \"langchain\",\n", 360 | " \"schema\",\n", 361 | " \"messages\",\n", 362 | " \"AIMessage\"\n", 363 | " ],\n", 364 | " \"kwargs\": {\n", 365 | " \"content\": \"\",\n", 366 | " \"additional_kwargs\": {\n", 367 | " \"function_call\": {\n", 368 | " \"arguments\": \"{\\n \\\"query\\\": \\\"weather in San Francisco\\\"\\n}\",\n", 369 | " \"name\": \"tavily_search_results_json\"\n", 370 | " }\n", 371 | " }\n", 372 | " }\n", 373 | " }\n", 374 | " ]\n", 375 | " }\n", 376 | "}\n", 377 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:RunnableSequence] [551ms] Exiting Chain run with output:\n", 378 | "\u001b[0m[outputs]\n", 379 | "\u001b[32;1m\u001b[1;3m[tool/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 9:tool:tavily_search_results_json] Entering Tool run with input:\n", 380 | "\u001b[0m\"{'query': 'weather in San Francisco'}\"\n", 381 | "\u001b[36;1m\u001b[1;3m[tool/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 9:tool:tavily_search_results_json] [3.43s] Exiting Tool run with output:\n", 382 | "\u001b[0m\"[{'url': 'https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/', 'content': 'Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in januaryWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 07-01-2023 54°F to 56°F. 08-01-2023 52°F to 58°F. 09-01-2023 54°F to ...'}, {'url': 'https://www.mercurynews.com/2024/01/07/bay-area-and-beyond-frost-advisory-and-freeze-warning-monday-morning/', 'content': 'degrees Fahrenheit respectively. In San Francisco, meanwhile, the low is forecasted to remain a relatively balmy 41 Weather | Bay Area will approach freezing temps overnight with light rain and large waves Weather | Weather | Bay Area temps could dip to freezing overnight after rain, heavy snowfall pound the Sierra Nevada Weather | Weather | 7 days of rain: How much did your city get during new year’s storms? Copyright © 2024 MediaNews GroupJanuary 7, 2024 at 10:49 a.m. After an unseasonably warm December, the Bay Area is ringing in the new year with freezing temperatures. According to the National Weather Service, interior parts of ...'}]\"\n", 383 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence] Entering Chain run with input:\n", 384 | "\u001b[0m[inputs]\n", 385 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 11:chain:RunnableAssign] Entering Chain run with input:\n", 386 | "\u001b[0m[inputs]\n", 387 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 11:chain:RunnableAssign > 12:chain:RunnableParallel] Entering Chain run with input:\n", 388 | "\u001b[0m[inputs]\n", 389 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 11:chain:RunnableAssign > 12:chain:RunnableParallel > 13:chain:] Entering Chain run with input:\n", 390 | "\u001b[0m[inputs]\n", 391 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 11:chain:RunnableAssign > 12:chain:RunnableParallel > 13:chain:] [0ms] Exiting Chain run with output:\n", 392 | "\u001b[0m{\n", 393 | " \"output\": [\n", 394 | " {\n", 395 | " \"lc\": 1,\n", 396 | " \"type\": \"constructor\",\n", 397 | " \"id\": [\n", 398 | " \"langchain\",\n", 399 | " \"schema\",\n", 400 | " \"messages\",\n", 401 | " \"AIMessage\"\n", 402 | " ],\n", 403 | " \"kwargs\": {\n", 404 | " \"content\": \"\",\n", 405 | " \"additional_kwargs\": {\n", 406 | " \"function_call\": {\n", 407 | " \"arguments\": \"{\\n \\\"query\\\": \\\"weather in San Francisco\\\"\\n}\",\n", 408 | " \"name\": \"tavily_search_results_json\"\n", 409 | " }\n", 410 | " }\n", 411 | " }\n", 412 | " },\n", 413 | " {\n", 414 | " \"lc\": 1,\n", 415 | " \"type\": \"constructor\",\n", 416 | " \"id\": [\n", 417 | " \"langchain\",\n", 418 | " \"schema\",\n", 419 | " \"messages\",\n", 420 | " \"FunctionMessage\"\n", 421 | " ],\n", 422 | " \"kwargs\": {\n", 423 | " \"name\": \"tavily_search_results_json\",\n", 424 | " \"content\": \"[{\\\"url\\\": \\\"https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/\\\", \\\"content\\\": \\\"Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in januaryWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 07-01-2023 54°F to 56°F. 08-01-2023 52°F to 58°F. 09-01-2023 54°F to ...\\\"}, {\\\"url\\\": \\\"https://www.mercurynews.com/2024/01/07/bay-area-and-beyond-frost-advisory-and-freeze-warning-monday-morning/\\\", \\\"content\\\": \\\"degrees Fahrenheit respectively. In San Francisco, meanwhile, the low is forecasted to remain a relatively balmy 41 Weather | Bay Area will approach freezing temps overnight with light rain and large waves Weather | Weather | Bay Area temps could dip to freezing overnight after rain, heavy snowfall pound the Sierra Nevada Weather | Weather | 7 days of rain: How much did your city get during new year’s storms? Copyright © 2024 MediaNews GroupJanuary 7, 2024 at 10:49 a.m. After an unseasonably warm December, the Bay Area is ringing in the new year with freezing temperatures. According to the National Weather Service, interior parts of ...\\\"}]\"\n", 425 | " }\n", 426 | " }\n", 427 | " ]\n", 428 | "}\n", 429 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 11:chain:RunnableAssign > 12:chain:RunnableParallel] [2ms] Exiting Chain run with output:\n", 430 | "\u001b[0m[outputs]\n", 431 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 11:chain:RunnableAssign] [3ms] Exiting Chain run with output:\n", 432 | "\u001b[0m{\n", 433 | " \"input\": \"what is the weather in sf?\",\n", 434 | " \"intermediate_steps\": [\n", 435 | " [\n", 436 | " {\n", 437 | " \"lc\": 1,\n", 438 | " \"type\": \"constructor\",\n", 439 | " \"id\": [\n", 440 | " \"langchain\",\n", 441 | " \"schema\",\n", 442 | " \"agent\",\n", 443 | " \"AgentActionMessageLog\"\n", 444 | " ],\n", 445 | " \"kwargs\": {\n", 446 | " \"tool\": \"tavily_search_results_json\",\n", 447 | " \"tool_input\": {\n", 448 | " \"query\": \"weather in San Francisco\"\n", 449 | " },\n", 450 | " \"log\": \"\\nInvoking: `tavily_search_results_json` with `{'query': 'weather in San Francisco'}`\\n\\n\\n\",\n", 451 | " \"message_log\": [\n", 452 | " {\n", 453 | " \"lc\": 1,\n", 454 | " \"type\": \"constructor\",\n", 455 | " \"id\": [\n", 456 | " \"langchain\",\n", 457 | " \"schema\",\n", 458 | " \"messages\",\n", 459 | " \"AIMessage\"\n", 460 | " ],\n", 461 | " \"kwargs\": {\n", 462 | " \"content\": \"\",\n", 463 | " \"additional_kwargs\": {\n", 464 | " \"function_call\": {\n", 465 | " \"arguments\": \"{\\n \\\"query\\\": \\\"weather in San Francisco\\\"\\n}\",\n", 466 | " \"name\": \"tavily_search_results_json\"\n", 467 | " }\n", 468 | " }\n", 469 | " }\n", 470 | " }\n", 471 | " ]\n", 472 | " }\n", 473 | " },\n", 474 | " [\n", 475 | " {\n", 476 | " \"url\": \"https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/\",\n", 477 | " \"content\": \"Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in januaryWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 07-01-2023 54°F to 56°F. 08-01-2023 52°F to 58°F. 09-01-2023 54°F to ...\"\n", 478 | " },\n", 479 | " {\n", 480 | " \"url\": \"https://www.mercurynews.com/2024/01/07/bay-area-and-beyond-frost-advisory-and-freeze-warning-monday-morning/\",\n", 481 | " \"content\": \"degrees Fahrenheit respectively. In San Francisco, meanwhile, the low is forecasted to remain a relatively balmy 41 Weather | Bay Area will approach freezing temps overnight with light rain and large waves Weather | Weather | Bay Area temps could dip to freezing overnight after rain, heavy snowfall pound the Sierra Nevada Weather | Weather | 7 days of rain: How much did your city get during new year’s storms? Copyright © 2024 MediaNews GroupJanuary 7, 2024 at 10:49 a.m. After an unseasonably warm December, the Bay Area is ringing in the new year with freezing temperatures. According to the National Weather Service, interior parts of ...\"\n", 482 | " }\n", 483 | " ]\n", 484 | " ]\n", 485 | " ],\n", 486 | " \"agent_scratchpad\": [\n", 487 | " {\n", 488 | " \"lc\": 1,\n", 489 | " \"type\": \"constructor\",\n", 490 | " \"id\": [\n", 491 | " \"langchain\",\n", 492 | " \"schema\",\n", 493 | " \"messages\",\n", 494 | " \"AIMessage\"\n", 495 | " ],\n", 496 | " \"kwargs\": {\n", 497 | " \"content\": \"\",\n", 498 | " \"additional_kwargs\": {\n", 499 | " \"function_call\": {\n", 500 | " \"arguments\": \"{\\n \\\"query\\\": \\\"weather in San Francisco\\\"\\n}\",\n", 501 | " \"name\": \"tavily_search_results_json\"\n", 502 | " }\n", 503 | " }\n", 504 | " }\n", 505 | " },\n", 506 | " {\n", 507 | " \"lc\": 1,\n", 508 | " \"type\": \"constructor\",\n", 509 | " \"id\": [\n", 510 | " \"langchain\",\n", 511 | " \"schema\",\n", 512 | " \"messages\",\n", 513 | " \"FunctionMessage\"\n", 514 | " ],\n", 515 | " \"kwargs\": {\n", 516 | " \"name\": \"tavily_search_results_json\",\n", 517 | " \"content\": \"[{\\\"url\\\": \\\"https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/\\\", \\\"content\\\": \\\"Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in januaryWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 07-01-2023 54°F to 56°F. 08-01-2023 52°F to 58°F. 09-01-2023 54°F to ...\\\"}, {\\\"url\\\": \\\"https://www.mercurynews.com/2024/01/07/bay-area-and-beyond-frost-advisory-and-freeze-warning-monday-morning/\\\", \\\"content\\\": \\\"degrees Fahrenheit respectively. In San Francisco, meanwhile, the low is forecasted to remain a relatively balmy 41 Weather | Bay Area will approach freezing temps overnight with light rain and large waves Weather | Weather | Bay Area temps could dip to freezing overnight after rain, heavy snowfall pound the Sierra Nevada Weather | Weather | 7 days of rain: How much did your city get during new year’s storms? Copyright © 2024 MediaNews GroupJanuary 7, 2024 at 10:49 a.m. After an unseasonably warm December, the Bay Area is ringing in the new year with freezing temperatures. According to the National Weather Service, interior parts of ...\\\"}]\"\n", 518 | " }\n", 519 | " }\n", 520 | " ]\n", 521 | "}\n", 522 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 14:prompt:ChatPromptTemplate] Entering Prompt run with input:\n", 523 | "\u001b[0m[inputs]\n", 524 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 14:prompt:ChatPromptTemplate] [1ms] Exiting Prompt run with output:\n", 525 | "\u001b[0m{\n", 526 | " \"lc\": 1,\n", 527 | " \"type\": \"constructor\",\n", 528 | " \"id\": [\n", 529 | " \"langchain\",\n", 530 | " \"prompts\",\n", 531 | " \"chat\",\n", 532 | " \"ChatPromptValue\"\n", 533 | " ],\n", 534 | " \"kwargs\": {\n", 535 | " \"messages\": [\n", 536 | " {\n", 537 | " \"lc\": 1,\n", 538 | " \"type\": \"constructor\",\n", 539 | " \"id\": [\n", 540 | " \"langchain\",\n", 541 | " \"schema\",\n", 542 | " \"messages\",\n", 543 | " \"SystemMessage\"\n", 544 | " ],\n", 545 | " \"kwargs\": {\n", 546 | " \"content\": \"You are a helpful assistant\",\n", 547 | " \"additional_kwargs\": {}\n", 548 | " }\n", 549 | " },\n", 550 | " {\n", 551 | " \"lc\": 1,\n", 552 | " \"type\": \"constructor\",\n", 553 | " \"id\": [\n", 554 | " \"langchain\",\n", 555 | " \"schema\",\n", 556 | " \"messages\",\n", 557 | " \"HumanMessage\"\n", 558 | " ],\n", 559 | " \"kwargs\": {\n", 560 | " \"content\": \"what is the weather in sf?\",\n", 561 | " \"additional_kwargs\": {}\n", 562 | " }\n", 563 | " },\n", 564 | " {\n", 565 | " \"lc\": 1,\n", 566 | " \"type\": \"constructor\",\n", 567 | " \"id\": [\n", 568 | " \"langchain\",\n", 569 | " \"schema\",\n", 570 | " \"messages\",\n", 571 | " \"AIMessage\"\n", 572 | " ],\n", 573 | " \"kwargs\": {\n", 574 | " \"content\": \"\",\n", 575 | " \"additional_kwargs\": {\n", 576 | " \"function_call\": {\n", 577 | " \"arguments\": \"{\\n \\\"query\\\": \\\"weather in San Francisco\\\"\\n}\",\n", 578 | " \"name\": \"tavily_search_results_json\"\n", 579 | " }\n", 580 | " }\n", 581 | " }\n", 582 | " },\n", 583 | " {\n", 584 | " \"lc\": 1,\n", 585 | " \"type\": \"constructor\",\n", 586 | " \"id\": [\n", 587 | " \"langchain\",\n", 588 | " \"schema\",\n", 589 | " \"messages\",\n", 590 | " \"FunctionMessage\"\n", 591 | " ],\n", 592 | " \"kwargs\": {\n", 593 | " \"name\": \"tavily_search_results_json\",\n", 594 | " \"content\": \"[{\\\"url\\\": \\\"https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/\\\", \\\"content\\\": \\\"Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in januaryWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 07-01-2023 54°F to 56°F. 08-01-2023 52°F to 58°F. 09-01-2023 54°F to ...\\\"}, {\\\"url\\\": \\\"https://www.mercurynews.com/2024/01/07/bay-area-and-beyond-frost-advisory-and-freeze-warning-monday-morning/\\\", \\\"content\\\": \\\"degrees Fahrenheit respectively. In San Francisco, meanwhile, the low is forecasted to remain a relatively balmy 41 Weather | Bay Area will approach freezing temps overnight with light rain and large waves Weather | Weather | Bay Area temps could dip to freezing overnight after rain, heavy snowfall pound the Sierra Nevada Weather | Weather | 7 days of rain: How much did your city get during new year’s storms? Copyright © 2024 MediaNews GroupJanuary 7, 2024 at 10:49 a.m. After an unseasonably warm December, the Bay Area is ringing in the new year with freezing temperatures. According to the National Weather Service, interior parts of ...\\\"}]\"\n", 595 | " }\n", 596 | " }\n", 597 | " ]\n", 598 | " }\n", 599 | "}\n", 600 | "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 15:llm:ChatOpenAI] Entering LLM run with input:\n", 601 | "\u001b[0m{\n", 602 | " \"prompts\": [\n", 603 | " \"System: You are a helpful assistant\\nHuman: what is the weather in sf?\\nAI: {'arguments': '{\\\\n \\\"query\\\": \\\"weather in San Francisco\\\"\\\\n}', 'name': 'tavily_search_results_json'}\\nFunction: [{\\\"url\\\": \\\"https://www.whereandwhen.net/when/north-america/california/san-francisco-ca/january/\\\", \\\"content\\\": \\\"Best time to go to San Francisco? Weather in San Francisco in january 2024 How was the weather last january? Here is the day by day recorded weather in San Francisco in january 2023: Seasonal average climate and temperature of San Francisco in january 8% 46% 29% 12% 8% Evolution of daily average temperature and precipitation in San Francisco in januaryWeather in San Francisco in january 2024. The weather in San Francisco in january comes from statistical datas on the past years. You can view the weather statistics the entire month, but also by using the tabs for the beginning, the middle and the end of the month. ... 07-01-2023 54°F to 56°F. 08-01-2023 52°F to 58°F. 09-01-2023 54°F to ...\\\"}, {\\\"url\\\": \\\"https://www.mercurynews.com/2024/01/07/bay-area-and-beyond-frost-advisory-and-freeze-warning-monday-morning/\\\", \\\"content\\\": \\\"degrees Fahrenheit respectively. In San Francisco, meanwhile, the low is forecasted to remain a relatively balmy 41 Weather | Bay Area will approach freezing temps overnight with light rain and large waves Weather | Weather | Bay Area temps could dip to freezing overnight after rain, heavy snowfall pound the Sierra Nevada Weather | Weather | 7 days of rain: How much did your city get during new year’s storms? Copyright © 2024 MediaNews GroupJanuary 7, 2024 at 10:49 a.m. After an unseasonably warm December, the Bay Area is ringing in the new year with freezing temperatures. According to the National Weather Service, interior parts of ...\\\"}]\"\n", 604 | " ]\n", 605 | "}\n", 606 | "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 15:llm:ChatOpenAI] [1.53s] Exiting LLM run with output:\n", 607 | "\u001b[0m{\n", 608 | " \"generations\": [\n", 609 | " [\n", 610 | " {\n", 611 | " \"text\": \"The weather in San Francisco is currently not available. However, according to past weather data, the average temperature in January ranges from 52°F to 58°F. Please note that weather conditions can vary, so it's always a good idea to check a reliable weather source for the most up-to-date information.\",\n", 612 | " \"generation_info\": {\n", 613 | " \"finish_reason\": \"stop\",\n", 614 | " \"logprobs\": null\n", 615 | " },\n", 616 | " \"type\": \"ChatGeneration\",\n", 617 | " \"message\": {\n", 618 | " \"lc\": 1,\n", 619 | " \"type\": \"constructor\",\n", 620 | " \"id\": [\n", 621 | " \"langchain\",\n", 622 | " \"schema\",\n", 623 | " \"messages\",\n", 624 | " \"AIMessage\"\n", 625 | " ],\n", 626 | " \"kwargs\": {\n", 627 | " \"content\": \"The weather in San Francisco is currently not available. However, according to past weather data, the average temperature in January ranges from 52°F to 58°F. Please note that weather conditions can vary, so it's always a good idea to check a reliable weather source for the most up-to-date information.\",\n", 628 | " \"additional_kwargs\": {}\n", 629 | " }\n", 630 | " }\n", 631 | " }\n", 632 | " ]\n", 633 | " ],\n", 634 | " \"llm_output\": {\n", 635 | " \"token_usage\": {\n", 636 | " \"completion_tokens\": 63,\n", 637 | " \"prompt_tokens\": 536,\n", 638 | " \"total_tokens\": 599\n", 639 | " },\n", 640 | " \"model_name\": \"gpt-3.5-turbo\",\n", 641 | " \"system_fingerprint\": null\n", 642 | " },\n", 643 | " \"run\": null\n", 644 | "}\n", 645 | "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 16:parser:OpenAIFunctionsAgentOutputParser] Entering Parser run with input:\n", 646 | "\u001b[0m[inputs]\n", 647 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence > 16:parser:OpenAIFunctionsAgentOutputParser] [1ms] Exiting Parser run with output:\n", 648 | "\u001b[0m{\n", 649 | " \"lc\": 1,\n", 650 | " \"type\": \"constructor\",\n", 651 | " \"id\": [\n", 652 | " \"langchain\",\n", 653 | " \"schema\",\n", 654 | " \"agent\",\n", 655 | " \"AgentFinish\"\n", 656 | " ],\n", 657 | " \"kwargs\": {\n", 658 | " \"return_values\": {\n", 659 | " \"output\": \"The weather in San Francisco is currently not available. However, according to past weather data, the average temperature in January ranges from 52°F to 58°F. Please note that weather conditions can vary, so it's always a good idea to check a reliable weather source for the most up-to-date information.\"\n", 660 | " },\n", 661 | " \"log\": \"The weather in San Francisco is currently not available. However, according to past weather data, the average temperature in January ranges from 52°F to 58°F. Please note that weather conditions can vary, so it's always a good idea to check a reliable weather source for the most up-to-date information.\"\n", 662 | " }\n", 663 | "}\n", 664 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 10:chain:RunnableSequence] [1.54s] Exiting Chain run with output:\n", 665 | "\u001b[0m[outputs]\n", 666 | "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor] [5.52s] Exiting Chain run with output:\n", 667 | "\u001b[0m{\n", 668 | " \"output\": \"The weather in San Francisco is currently not available. However, according to past weather data, the average temperature in January ranges from 52°F to 58°F. Please note that weather conditions can vary, so it's always a good idea to check a reliable weather source for the most up-to-date information.\"\n", 669 | "}\n" 670 | ] 671 | }, 672 | { 673 | "data": { 674 | "text/plain": [ 675 | "{'input': 'what is the weather in sf?',\n", 676 | " 'output': \"The weather in San Francisco is currently not available. However, according to past weather data, the average temperature in January ranges from 52°F to 58°F. Please note that weather conditions can vary, so it's always a good idea to check a reliable weather source for the most up-to-date information.\"}" 677 | ] 678 | }, 679 | "execution_count": 6, 680 | "metadata": {}, 681 | "output_type": "execute_result" 682 | } 683 | ], 684 | "source": [ 685 | "agent_executor.invoke({\"input\": \"what is the weather in sf?\"})" 686 | ] 687 | }, 688 | { 689 | "cell_type": "markdown", 690 | "id": "30270f35", 691 | "metadata": {}, 692 | "source": [ 693 | "## LangSmith" 694 | ] 695 | }, 696 | { 697 | "cell_type": "code", 698 | "execution_count": 7, 699 | "id": "297ea8e8", 700 | "metadata": {}, 701 | "outputs": [ 702 | { 703 | "name": "stdout", 704 | "output_type": "stream", 705 | "text": [ 706 | "········\n" 707 | ] 708 | } 709 | ], 710 | "source": [ 711 | "import os\n", 712 | "import getpass\n", 713 | "\n", 714 | "os.environ[\"LANGCHAIN_TRACING_V2\"]=\"true\"\n", 715 | "os.environ[\"LANGCHAIN_ENDPOINT\"]=\"https://api.smith.langchain.com\"\n", 716 | "os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()" 717 | ] 718 | }, 719 | { 720 | "cell_type": "code", 721 | "execution_count": 8, 722 | "id": "35ba1fa4", 723 | "metadata": {}, 724 | "outputs": [ 725 | { 726 | "data": { 727 | "text/plain": [ 728 | "{'input': 'what is the weather in sf?',\n", 729 | " 'output': \"I'm sorry, but I couldn't find the current weather in San Francisco. However, you can check the weather forecast for San Francisco on websites like Weather.com or AccuWeather.com.\"}" 730 | ] 731 | }, 732 | "execution_count": 8, 733 | "metadata": {}, 734 | "output_type": "execute_result" 735 | } 736 | ], 737 | "source": [ 738 | "set_debug(False)\n", 739 | "\n", 740 | "agent_executor.invoke({\"input\": \"what is the weather in sf?\"})" 741 | ] 742 | }, 743 | { 744 | "cell_type": "code", 745 | "execution_count": null, 746 | "id": "bbf926a5", 747 | "metadata": {}, 748 | "outputs": [], 749 | "source": [] 750 | } 751 | ], 752 | "metadata": { 753 | "kernelspec": { 754 | "display_name": "Python 3 (ipykernel)", 755 | "language": "python", 756 | "name": "python3" 757 | }, 758 | "language_info": { 759 | "codemirror_mode": { 760 | "name": "ipython", 761 | "version": 3 762 | }, 763 | "file_extension": ".py", 764 | "mimetype": "text/x-python", 765 | "name": "python", 766 | "nbconvert_exporter": "python", 767 | "pygments_lexer": "ipython3", 768 | "version": "3.11.1" 769 | } 770 | }, 771 | "nbformat": 4, 772 | "nbformat_minor": 5 773 | } 774 | -------------------------------------------------------------------------------- /output_parsers.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "485f3c32-bc94-4b2a-bbe9-366e381242a4", 6 | "metadata": {}, 7 | "source": [ 8 | "# Output Parsing\n", 9 | "\n", 10 | "This notebook covers functionality related to output parsing. For more information, see:\n", 11 | "\n", 12 | "- [List of all Output Parsers](https://python.langchain.com/docs/modules/model_io/output_parsers/)\n", 13 | "\n", 14 | "- [OpenAI Function Calling](https://python.langchain.com/docs/modules/model_io/output_parsers/types/openai_functions)" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "id": "baefaf66", 20 | "metadata": {}, 21 | "source": [ 22 | "## Converting Messages" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 1, 28 | "id": "1a53c356", 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "from langchain_openai import ChatOpenAI\n", 33 | "from langchain_core.prompts import ChatPromptTemplate" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": 2, 39 | "id": "7db51d8e", 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "prompt = ChatPromptTemplate.from_template(\"Tell me a joke about {topic}\")\n", 44 | "model = ChatOpenAI()" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": 3, 50 | "id": "7fbd6d14", 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "chain = prompt | model" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": 4, 60 | "id": "9b028d5e", 61 | "metadata": {}, 62 | "outputs": [ 63 | { 64 | "data": { 65 | "text/plain": [ 66 | "AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\")" 67 | ] 68 | }, 69 | "execution_count": 4, 70 | "metadata": {}, 71 | "output_type": "execute_result" 72 | } 73 | ], 74 | "source": [ 75 | "chain.invoke({\"topic\": \"bears\"})" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 5, 81 | "id": "f31fd201", 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "from langchain_core.output_parsers import StrOutputParser\n", 86 | "\n", 87 | "parser = StrOutputParser()\n", 88 | "chain = prompt | model | parser" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": 6, 94 | "id": "86037589", 95 | "metadata": {}, 96 | "outputs": [ 97 | { 98 | "data": { 99 | "text/plain": [ 100 | "\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\"" 101 | ] 102 | }, 103 | "execution_count": 6, 104 | "metadata": {}, 105 | "output_type": "execute_result" 106 | } 107 | ], 108 | "source": [ 109 | "chain.invoke({\"topic\": \"bears\"})" 110 | ] 111 | }, 112 | { 113 | "cell_type": "markdown", 114 | "id": "620eb059", 115 | "metadata": {}, 116 | "source": [ 117 | "## OpenAI Function Calling" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": 14, 123 | "id": "f9e2cb72", 124 | "metadata": {}, 125 | "outputs": [], 126 | "source": [ 127 | "from langchain.utils.openai_functions import (\n", 128 | " convert_pydantic_to_openai_function,\n", 129 | ")\n", 130 | "from langchain_core.prompts import ChatPromptTemplate\n", 131 | "from langchain_core.pydantic_v1 import BaseModel, Field, validator" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": 8, 137 | "id": "052534a4", 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [ 141 | "class Joke(BaseModel):\n", 142 | " \"\"\"Joke to tell user.\"\"\"\n", 143 | "\n", 144 | " setup: str = Field(description=\"question to set up a joke\")\n", 145 | " punchline: str = Field(description=\"answer to resolve the joke\")\n", 146 | "\n", 147 | "\n", 148 | "openai_functions = [convert_pydantic_to_openai_function(Joke)]" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": 9, 154 | "id": "27be9acc", 155 | "metadata": {}, 156 | "outputs": [], 157 | "source": [ 158 | "from langchain.output_parsers.openai_functions import JsonOutputFunctionsParser" 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": 10, 164 | "id": "8b14e0fb", 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "parser = JsonOutputFunctionsParser()" 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": 11, 174 | "id": "d51ea7df", 175 | "metadata": {}, 176 | "outputs": [], 177 | "source": [ 178 | "chain = prompt | model.bind(functions=openai_functions) | parser" 179 | ] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "execution_count": 12, 184 | "id": "8acc8289", 185 | "metadata": {}, 186 | "outputs": [ 187 | { 188 | "data": { 189 | "text/plain": [ 190 | "{'setup': \"Why don't bears wear shoes?\",\n", 191 | " 'punchline': 'Because they have bear feet!'}" 192 | ] 193 | }, 194 | "execution_count": 12, 195 | "metadata": {}, 196 | "output_type": "execute_result" 197 | } 198 | ], 199 | "source": [ 200 | "chain.invoke({\"topic\": \"bears\"})" 201 | ] 202 | }, 203 | { 204 | "cell_type": "markdown", 205 | "id": "660537da", 206 | "metadata": {}, 207 | "source": [ 208 | "## Streaming" 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": 13, 214 | "id": "346804c2", 215 | "metadata": {}, 216 | "outputs": [ 217 | { 218 | "name": "stdout", 219 | "output_type": "stream", 220 | "text": [ 221 | "{}\n", 222 | "{'setup': ''}\n", 223 | "{'setup': 'Why'}\n", 224 | "{'setup': 'Why don'}\n", 225 | "{'setup': \"Why don't\"}\n", 226 | "{'setup': \"Why don't scientists\"}\n", 227 | "{'setup': \"Why don't scientists trust\"}\n", 228 | "{'setup': \"Why don't scientists trust atoms\"}\n", 229 | "{'setup': \"Why don't scientists trust atoms?\"}\n", 230 | "{'setup': \"Why don't scientists trust atoms?\", 'punchline': ''}\n", 231 | "{'setup': \"Why don't scientists trust atoms?\", 'punchline': 'Because'}\n", 232 | "{'setup': \"Why don't scientists trust atoms?\", 'punchline': 'Because they'}\n", 233 | "{'setup': \"Why don't scientists trust atoms?\", 'punchline': 'Because they make'}\n", 234 | "{'setup': \"Why don't scientists trust atoms?\", 'punchline': 'Because they make up'}\n", 235 | "{'setup': \"Why don't scientists trust atoms?\", 'punchline': 'Because they make up everything'}\n", 236 | "{'setup': \"Why don't scientists trust atoms?\", 'punchline': 'Because they make up everything!'}\n" 237 | ] 238 | } 239 | ], 240 | "source": [ 241 | "for s in chain.stream({\"topic\": \"bars\"}):\n", 242 | " print(s)" 243 | ] 244 | } 245 | ], 246 | "metadata": { 247 | "kernelspec": { 248 | "display_name": "Python 3 (ipykernel)", 249 | "language": "python", 250 | "name": "python3" 251 | }, 252 | "language_info": { 253 | "codemirror_mode": { 254 | "name": "ipython", 255 | "version": 3 256 | }, 257 | "file_extension": ".py", 258 | "mimetype": "text/x-python", 259 | "name": "python", 260 | "nbconvert_exporter": "python", 261 | "pygments_lexer": "ipython3", 262 | "version": "3.11.1" 263 | } 264 | }, 265 | "nbformat": 4, 266 | "nbformat_minor": 5 267 | } 268 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | # LangChain requirements 2 | langchain>=0.1 3 | langchain_openai 4 | langchainhub 5 | tavily-python 6 | faiss-cpu 7 | 8 | # Dev requirements 9 | jupyter 10 | -------------------------------------------------------------------------------- /retrieval.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "376c865a-9f50-417d-a0cf-4c430571d104", 6 | "metadata": {}, 7 | "source": [ 8 | "# Retrieval\n", 9 | "\n", 10 | "This notebook covers basic walkthrough of retrieval functionality in LangChain. For more information, see:\n", 11 | "\n", 12 | "- [Retrieval Documentation](https://python.langchain.com/docs/modules/data_connection/)\n", 13 | "\n", 14 | "- [Advanced Retrieval Types](https://python.langchain.com/docs/modules/data_connection/retrievers/)\n", 15 | "\n", 16 | "- [QA with RAG Use Case Documentation](https://python.langchain.com/docs/use_cases/question_answering/)" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "49b594de", 22 | "metadata": {}, 23 | "source": [ 24 | "## Load Documents" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": 1, 30 | "id": "16bda46c", 31 | "metadata": {}, 32 | "outputs": [], 33 | "source": [ 34 | "from langchain_community.document_loaders import WebBaseLoader\n", 35 | "loader = WebBaseLoader(\"https://docs.smith.langchain.com/overview\")\n", 36 | "\n", 37 | "docs = loader.load()" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": 2, 43 | "id": "fb65747c", 44 | "metadata": {}, 45 | "outputs": [ 46 | { 47 | "data": { 48 | "text/plain": [ 49 | "1" 50 | ] 51 | }, 52 | "execution_count": 2, 53 | "metadata": {}, 54 | "output_type": "execute_result" 55 | } 56 | ], 57 | "source": [ 58 | "len(docs)" 59 | ] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "id": "8aece40e", 64 | "metadata": {}, 65 | "source": [ 66 | "## Split documents" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "execution_count": 3, 72 | "id": "8e749d8a", 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", 77 | "\n", 78 | "\n", 79 | "text_splitter = RecursiveCharacterTextSplitter()\n", 80 | "documents = text_splitter.split_documents(docs)" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": 4, 86 | "id": "006a257f", 87 | "metadata": {}, 88 | "outputs": [ 89 | { 90 | "data": { 91 | "text/plain": [ 92 | "5" 93 | ] 94 | }, 95 | "execution_count": 4, 96 | "metadata": {}, 97 | "output_type": "execute_result" 98 | } 99 | ], 100 | "source": [ 101 | "len(documents)" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "id": "792485da", 107 | "metadata": {}, 108 | "source": [ 109 | "## Index Documents" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": 12, 115 | "id": "289e53d3", 116 | "metadata": {}, 117 | "outputs": [], 118 | "source": [ 119 | "from langchain_openai import OpenAIEmbeddings\n", 120 | "from langchain_community.vectorstores import FAISS\n", 121 | "\n", 122 | "embeddings = OpenAIEmbeddings()\n", 123 | "vector = FAISS.from_documents(documents, embeddings)" 124 | ] 125 | }, 126 | { 127 | "cell_type": "markdown", 128 | "id": "ad7d2450", 129 | "metadata": {}, 130 | "source": [ 131 | "## Query Documents\n" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": 13, 137 | "id": "85a66449", 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [ 141 | "from langchain.chains.combine_documents import create_stuff_documents_chain\n", 142 | "from langchain_openai import ChatOpenAI\n", 143 | "from langchain_core.prompts import ChatPromptTemplate\n", 144 | "\n", 145 | "prompt = ChatPromptTemplate.from_template(\"\"\"Answer the following question based only on the provided context:\n", 146 | "\n", 147 | "\n", 148 | "{context}\n", 149 | "\n", 150 | "\n", 151 | "Question: {input}\"\"\")\n", 152 | "llm = ChatOpenAI()\n", 153 | "\n", 154 | "document_chain = create_stuff_documents_chain(llm, prompt)" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": 14, 160 | "id": "588f5818", 161 | "metadata": {}, 162 | "outputs": [], 163 | "source": [ 164 | "from langchain.chains import create_retrieval_chain\n", 165 | "\n", 166 | "retriever = vector.as_retriever()\n", 167 | "retrieval_chain = create_retrieval_chain(retriever, document_chain)" 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": 15, 173 | "id": "93cfd3ff", 174 | "metadata": {}, 175 | "outputs": [ 176 | { 177 | "name": "stdout", 178 | "output_type": "stream", 179 | "text": [ 180 | "LangSmith can help with testing by providing a way to run chains over datasets and visualize the outputs. It simplifies dataset uploading and allows users to review and evaluate the results. The LangSmith client makes it easy to pull down a dataset and run a chain over the data points, logging the results to a new project associated with the dataset. Users can assign feedback to runs and mark them as correct or incorrect directly in the web app, displaying aggregate statistics for each test project. LangSmith also provides evaluators that can be specified when initiating a test run to evaluate the results. Additionally, LangSmith offers annotation queues for manual review and annotation of runs, allowing users to assess subjective qualities and validate auto-evaluated runs.\n" 181 | ] 182 | } 183 | ], 184 | "source": [ 185 | "response = retrieval_chain.invoke({\"input\": \"how can langsmith help with testing?\"})\n", 186 | "print(response[\"answer\"])" 187 | ] 188 | }, 189 | { 190 | "cell_type": "markdown", 191 | "id": "8c9ba679", 192 | "metadata": {}, 193 | "source": [ 194 | "## Advanced Retrieval" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": 16, 200 | "id": "52d89b08", 201 | "metadata": {}, 202 | "outputs": [], 203 | "source": [ 204 | "from langchain.retrievers import MultiQueryRetriever\n", 205 | "\n", 206 | "advanced_retriever = MultiQueryRetriever.from_llm(retriever=retriever, llm=llm)" 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "execution_count": 17, 212 | "id": "67347a0f", 213 | "metadata": {}, 214 | "outputs": [], 215 | "source": [ 216 | "retrieval_chain = create_retrieval_chain(advanced_retriever, document_chain)" 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": 18, 222 | "id": "685ccfb9", 223 | "metadata": {}, 224 | "outputs": [ 225 | { 226 | "name": "stderr", 227 | "output_type": "stream", 228 | "text": [ 229 | "/Users/harrisonchase/.pyenv/versions/3.11.1/envs/langchain-0.1-guides/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:189: LangChainDeprecationWarning: The function `__call__` was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use invoke instead.\n", 230 | " warn_deprecated(\n" 231 | ] 232 | }, 233 | { 234 | "name": "stdout", 235 | "output_type": "stream", 236 | "text": [ 237 | "LangSmith can help with testing by providing the following features:\n", 238 | "\n", 239 | "1. Visualization of Inputs/Outputs: LangSmith provides a straightforward visualization of the exact inputs and outputs to all LLM calls, allowing users to easily understand them.\n", 240 | "\n", 241 | "2. Prompt Editing Playground: LangSmith has a built-in playground where users can modify the prompt of an LLM call and re-run it to observe the resulting changes to the output. This feature supports OpenAI and Anthropic models and works for LLM and Chat Model calls.\n", 242 | "\n", 243 | "3. Sequence of Events: LangSmith's tracing feature offers a visualization of the sequence of calls and interactions in complicated chains and agents. This helps users understand the exact sequence of events and the inputs/outputs of each call.\n", 244 | "\n", 245 | "4. Latency Monitoring: LangSmith tracks the latency of each step in a chain, allowing users to identify and potentially eliminate the slowest components that may be causing a chain to take longer than expected.\n", 246 | "\n", 247 | "5. Token Usage Tracking: LangSmith tracks the total token usage for a chain and the token usage of each step. This helps users identify potentially costly parts of the chain and optimize their token usage.\n", 248 | "\n", 249 | "6. Collaborative Debugging: LangSmith provides a \"Share\" button that allows users to share a faulty chain and its LLM runs with colleagues for collaborative debugging. This makes it easier to work together on identifying and resolving issues.\n", 250 | "\n", 251 | "7. Collecting Examples: LangSmith includes an \"Add to Dataset\" button for each run, making it easy to add input/output examples to a chosen dataset. Users can edit the examples before adding them, including the expected result. This feature helps in collecting examples of failures and known issues for testing future chain versions.\n", 252 | "\n", 253 | "Overall, LangSmith provides various tools and features to facilitate testing and evaluation of LLM applications, improving the reliability and quality of the models.\n" 254 | ] 255 | } 256 | ], 257 | "source": [ 258 | "response = retrieval_chain.invoke({\"input\": \"how can langsmith help with testing?\"})\n", 259 | "print(response[\"answer\"])" 260 | ] 261 | }, 262 | { 263 | "cell_type": "code", 264 | "execution_count": null, 265 | "id": "80e992b4", 266 | "metadata": {}, 267 | "outputs": [], 268 | "source": [] 269 | } 270 | ], 271 | "metadata": { 272 | "kernelspec": { 273 | "display_name": "Python 3 (ipykernel)", 274 | "language": "python", 275 | "name": "python3" 276 | }, 277 | "language_info": { 278 | "codemirror_mode": { 279 | "name": "ipython", 280 | "version": 3 281 | }, 282 | "file_extension": ".py", 283 | "mimetype": "text/x-python", 284 | "name": "python", 285 | "nbconvert_exporter": "python", 286 | "pygments_lexer": "ipython3", 287 | "version": "3.11.1" 288 | } 289 | }, 290 | "nbformat": 4, 291 | "nbformat_minor": 5 292 | } 293 | --------------------------------------------------------------------------------