├── 03_AI_agents ├── .DS_Store ├── Lab2 │ ├── Chinook.db │ └── README.md ├── Lab3 │ └── README.md └── Lab1 │ └── README.md ├── 02_Vector database & RAG ├── .DS_Store ├── Lab1 │ ├── .DS_Store │ ├── kodeks_pracy.pdf │ ├── praca_rodzicielstwo.pdf │ └── README.md ├── Lab3 │ └── README.md └── Lab2 │ └── README.md ├── 04_Multi-agent systems ├── Lab2 │ ├── graph.jpeg │ └── README.md └── Lab1 │ └── README.md ├── README.md └── 01_Langchain basics ├── Lab3 └── README.md ├── Lab2 └── README.md └── Lab1 └── README.md /03_AI_agents/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kali11/AI-agent-workshops/HEAD/03_AI_agents/.DS_Store -------------------------------------------------------------------------------- /03_AI_agents/Lab2/Chinook.db: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kali11/AI-agent-workshops/HEAD/03_AI_agents/Lab2/Chinook.db -------------------------------------------------------------------------------- /02_Vector database & RAG/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kali11/AI-agent-workshops/HEAD/02_Vector database & RAG/.DS_Store -------------------------------------------------------------------------------- /04_Multi-agent systems/Lab2/graph.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kali11/AI-agent-workshops/HEAD/04_Multi-agent systems/Lab2/graph.jpeg -------------------------------------------------------------------------------- /02_Vector database & RAG/Lab1/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kali11/AI-agent-workshops/HEAD/02_Vector database & RAG/Lab1/.DS_Store -------------------------------------------------------------------------------- /02_Vector database & RAG/Lab1/kodeks_pracy.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kali11/AI-agent-workshops/HEAD/02_Vector database & RAG/Lab1/kodeks_pracy.pdf -------------------------------------------------------------------------------- /02_Vector database & RAG/Lab1/praca_rodzicielstwo.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kali11/AI-agent-workshops/HEAD/02_Vector database & RAG/Lab1/praca_rodzicielstwo.pdf -------------------------------------------------------------------------------- /03_AI_agents/Lab3/README.md: -------------------------------------------------------------------------------- 1 | # Lab 3: TASK: Build your AI agent! 2 | 3 | In this lab you build your own AI agent. 4 | 5 | ## Prerequisutes: 6 | - OpenAI API key with a few Euros credits 7 | - Google account 8 | - Tavily API key 9 | 10 | ## Task description 11 | Build an agent that will be capable of generating images using DALL-E. Here is a sample prompt: 12 | 13 | ```python 14 | agent_executor.invoke( 15 | { 16 | "input": "Find a detailed description of a Mermaid of Warsaw and generate a dall-e prompt. Tweak the prompt and generate an image" 17 | } 18 | ) 19 | ``` -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AI-agent-workshops 2 | A repository with labs for AI agent workshops. 3 | 4 | More details: 5 | https://ai72h.piotrkalinowski.cloud/ 6 | 7 | All labs are self-paced. You can do them in any order however I recommend to start with 01_Langchain_basics / Lab 1. 8 | 9 | ### Workshops content: 10 | 11 | 1. 01_Langchain_basics 12 | - Lab 1 - building chains using LCEL (Langchain Expression Language) 13 | - Lab 2 - building LinkedIn post generator 14 | - Lab 3 - building chatbot 15 | 2. 02_Vector database & RAG 16 | - Lab 1 - Embeddings and loading documents into vector database 17 | - Lab 2 - building RAG (Retrieval Augmented Generation) 18 | - Lab 3 - building RAG agent 19 | 3. 03_AI_agents 20 | - Lab 1 - creating first agent 21 | - Lab 2 - building mulit-tools agent 22 | - Lab 3 - HOMEWORK 23 | 4. 04_Multi-agent systems 24 | - Lab 1 - building multi-agent system with AutoGen 25 | - Lab 1 - building multi-agent system with Langgraph 26 | 27 | 28 | In case of any question contact me at: 29 | [kontakt@piotrkalinowski.cloud](mailto:kontakt@piotrkalinowski.cloud) 30 | -------------------------------------------------------------------------------- /02_Vector database & RAG/Lab3/README.md: -------------------------------------------------------------------------------- 1 | # Lab 3: RAG Agent 2 | In this lab we will build an AI agent with RAG capabilities. This lab creates a simple agent, more sophisticated agents are build during Day 3. 3 | 4 | ## Prerequisutes: 5 | - OpenAI API key with a few Euros credits 6 | - Google account 7 | - Lab1 finished 8 | 9 | ## Task 1: Set-up 10 | 1. Open Google Colab: https://colab.research.google.com/ 11 | 2. Create new notebook, name it eg. **Workshop2 - la3** 12 | 3. First, we need to install dependencies. In the first cell type and run: 13 | 14 | ```python 15 | !pip install --quiet langchain==0.2.16 langchain-openai==0.1.23 langchain-community==0.2.16 qdrant-client==1.12.0 langchainhub==0.1.21 16 | ``` 17 | 18 | 4. Connecto to LansSmith for debuggin purposes: 19 | 20 | ```python 21 | import os 22 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 23 | os.environ["LANGCHAIN_PROJECT"] = "ai-workshops-day2" 24 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 25 | os.environ["LANGCHAIN_API_KEY"] = "" 26 | ``` 27 | 28 | 5. Configure Qdrant client, embeddings model and LLM: 29 | 30 | ```python 31 | import os 32 | from qdrant_client import QdrantClient 33 | from langchain_openai import OpenAIEmbeddings 34 | from google.colab import userdata 35 | from langchain_community.vectorstores import Qdrant 36 | from langchain_openai import ChatOpenAI 37 | 38 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 39 | 40 | embeddings = OpenAIEmbeddings(model = "text-embedding-3-large") 41 | collection_name = "labor_law" 42 | 43 | qdrant_client = QdrantClient( 44 | url="", 45 | api_key=userdata.get('qdrant_key') 46 | ) 47 | qdrant = Qdrant(qdrant_client, collection_name, embeddings) 48 | 49 | gpt4 = ChatOpenAI(model = "gpt-4o") 50 | ``` 51 | 52 | ## Task 2: Create the agent 53 | 1. Create retriever: 54 | ```python 55 | retriever = qdrant.as_retriever() 56 | ``` 57 | 58 | 2. Create a retriever tool for the agent: 59 | ```python 60 | from langchain.tools.retriever import create_retriever_tool 61 | 62 | retriever_tool = create_retriever_tool( 63 | retriever, 64 | "search_documents", 65 | "Searches and returns documents about labor law in Poland", 66 | ) 67 | tools = [retriever_tool] 68 | ``` 69 | 70 | 3. Create a prompt. Here we will use [langchain hub](https://smith.langchain.com/hub), which is a place where you can find different, publicly available prompts. Go to hub and see how **hwchase17/openai-tools-agent** looks like. 71 | 72 | ```python 73 | from langchain import hub 74 | 75 | prompt = hub.pull("hwchase17/openai-tools-agent") 76 | ``` 77 | 78 | 4. Now, let's create the agent with tools and the agent executor: 79 | ```python 80 | from langchain.agents import AgentExecutor, create_tool_calling_agent 81 | 82 | agent = create_tool_calling_agent(gpt4, tools, prompt) 83 | agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True) 84 | ``` 85 | 86 | 5. Invoke the agent and observe the output: 87 | 88 | ```python 89 | agent_executor.invoke( 90 | { 91 | "input": "Ile przysługuje dni urlopu wypoczynkowego?" 92 | } 93 | ) 94 | ``` 95 | 96 | ## End lab -------------------------------------------------------------------------------- /01_Langchain basics/Lab3/README.md: -------------------------------------------------------------------------------- 1 | # Lab 3: Chatbot with memory 2 | In this lab you will build a chatbot that remembers conversation history. A chatbot is basically a chain but with memory. 3 | 4 | ## Prerequisutes: 5 | - OpenAI API key with a few Euros credits 6 | - Google account 7 | 8 | ## Task 1: Set-up 9 | 1. Open Google Colab: https://colab.research.google.com/ 10 | 1. Create new notebook, name it eg. **Workshop1 - lab3** 11 | 1. First, we need to install dependencies. In the first cell type and run: 12 | 13 | ```python 14 | !pip install --quiet langchain==0.2.16 langchain-openai==0.1.23 langchain-community==0.2.16 15 | ``` 16 | 17 | Here we install Langchain framework and langchain-openai responsible for OpenAI integration. 18 | 1. In the next cell create an instance of gpt-4o model: 19 | 20 | ```python 21 | import os 22 | from langchain_openai import ChatOpenAI 23 | from google.colab import userdata 24 | 25 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 26 | 27 | gpt4 = ChatOpenAI(model = "gpt-4o-mini") 28 | ``` 29 | 30 | 2. Switch on LangSmith: 31 | 32 | ```python 33 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 34 | os.environ["LANGCHAIN_PROJECT"] = "ai-agent-workshops" 35 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 36 | os.environ["LANGCHAIN_API_KEY"] = "YOUR_LANGSMITH_KEY" 37 | ``` 38 | 39 | ## Task 2: Build a simple chatbot 40 | LLM models and thier's APIs are usually stateless. That means we need to always send to the model whole message history. Let's see an example: 41 | 1. Create new cell and paste: 42 | 43 | ```python 44 | from langchain_core.messages import HumanMessage 45 | 46 | gpt4.invoke( 47 | [ 48 | HumanMessage( 49 | content="What is the capital of Poland?" 50 | ) 51 | ] 52 | ) 53 | ``` 54 | 55 | You should see the result in the output as an instance of AIMessage. 56 | 57 | 2. But we don't have a history: 58 | 59 | ```python 60 | gpt4.invoke([HumanMessage(content="What I was asking about?")]) 61 | ``` 62 | 63 | 3. We need to supply whole history: 64 | 65 | ```python 66 | from langchain_core.messages import AIMessage 67 | 68 | gpt4.invoke( 69 | [ 70 | HumanMessage( 71 | content="What is the capital of Poland?" 72 | ), 73 | AIMessage(content="The capital of Poland is Warsaw"), 74 | HumanMessage(content="What I was asking about?") 75 | ] 76 | ) 77 | ``` 78 | 79 | 4. We can also make it more effective using **MessagesPlaceholder**. This class is very useful when constructing prompts: 80 | ```python 81 | from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder 82 | 83 | prompt = ChatPromptTemplate.from_messages( 84 | [ 85 | ( 86 | "system", 87 | "You are a helpful assistant.", 88 | ), 89 | MessagesPlaceholder(variable_name="messages"), 90 | ] 91 | ) 92 | 93 | chain = prompt | gpt4 94 | ``` 95 | 96 | ## Task 3: Add memory 97 | Langchain comes with many different memory types. Memory can be: 98 | - handled by ourselves - then we need to build it manually 99 | - handled by Langchain 100 | 101 | 1. First, let's do it manually. In a new cell type: 102 | 103 | ```python 104 | from langchain.memory import ChatMessageHistory 105 | 106 | chat_history = ChatMessageHistory() 107 | chat_history.add_user_message("What is the capital of Poland?") 108 | chat_history.add_ai_message("The capital of Poland is Warsaw") 109 | chat_history.add_user_message("How large is this city?") 110 | chat_history.messages 111 | ``` 112 | 113 | 2. And let's invoke: 114 | 115 | ```python 116 | chain.invoke({"messages": chat_history.messages}) 117 | ``` 118 | 119 | 3. Now, the memory will be handled by Langchain. **RunnableWithMessageHistory** is a class that allows us to use memory with Runnables. A few remarks here: 120 | - **RunnableWithMessageHistory** encloses a chain 121 | - second argument is a function that returns chat history 122 | - third argument is a key for input. It needs to match the placeholder in a promp template 123 | - fourth argument is a key for history. Again, it needs to match the placeholder in a promp template 124 | 125 | ```python 126 | from langchain_core.runnables.history import RunnableWithMessageHistory 127 | 128 | chat_history = ChatMessageHistory() 129 | 130 | prompt = ChatPromptTemplate.from_messages( 131 | [ 132 | ( 133 | "system", 134 | "You are a helpful assistant.", 135 | ), 136 | MessagesPlaceholder(variable_name="chat_history"), 137 | ("human", "{input}"), 138 | ] 139 | ) 140 | 141 | chain = prompt | gpt4 142 | 143 | chain_with_history = RunnableWithMessageHistory( 144 | chain, 145 | lambda session_id: chat_history, 146 | input_messages_key="input", 147 | history_messages_key="chat_history", 148 | ) 149 | ``` 150 | 151 | 4. Now let's invoke our chain. We are adding a "configurable" key to pass session_id. It is unused, but it is required. 152 | 153 | ```python 154 | chain_with_history.invoke( 155 | {"input": "What is the capital of Poland"}, 156 | {"configurable": {"session_id": "unused"}}, 157 | ) 158 | ``` 159 | 160 | 5. And one more time: 161 | 162 | ```python 163 | chain_with_history.invoke( 164 | {"input": "How large is this town?"}, 165 | {"configurable": {"session_id": "unused"}}, 166 | ) 167 | ``` 168 | 169 | 6. Observe the messages history: 170 | 171 | ```python 172 | chat_history.messages 173 | ``` 174 | 175 | ## Task 4: Use pre-defined Langchain chains. 176 | Langchain comes with pre-defined chains. They are ready to use, with default prompting already defined. **Some of them already are or will be depricated!** All of them can be found here: [chains](https://python.langchain.com/v0.1/docs/modules/chains/) 177 | 178 | > Langchain rather withdraws from those chains in favor of LCEL chains and Langgraph (Langchain's extension). You can learn more about Langgrpah in Day 4 of this workshops. 179 | 180 | 1. Let's use the LLMMathChain. It is simple chain optimized for mathematical operations. 181 | 182 | ```python 183 | from langchain.chains import LLMMathChain 184 | 185 | llm_math = LLMMathChain.from_llm(gpt4, verbose=True) 186 | ``` 187 | 188 | 2. Let's test: 189 | 190 | ```python 191 | llm_math.invoke(input="What is 9 raised to .9876 ower?") 192 | ``` 193 | 194 | 3. Check Langgraph and see how to chain works under the hood. 195 | 196 | ## END LAB 197 | -------------------------------------------------------------------------------- /04_Multi-agent systems/Lab1/README.md: -------------------------------------------------------------------------------- 1 | # Lab 1: Building multi-agent app with AutoGen 2 | In this lab you will learn how to build a multi-agent app using AutoGen framework 3 | 4 | ## Prerequisutes: 5 | - OpenAI API key with a few Euros credits 6 | - Google account 7 | 8 | ## Task 1: Set-up 9 | 1. Open Google Colab: https://colab.research.google.com/ 10 | 2. Create new notebook, name it eg. **Workshop4 - la1** 11 | 3. First, we need to install dependencies. In the first cell type and run: 12 | 13 | ```python 14 | !pip install "pyautogen>=0.2.18" 15 | ``` 16 | 17 | 4. Create a configuration for all AI agents: 18 | 19 | ```python 20 | from google.colab import userdata 21 | 22 | config = { 23 | "config_list": [ 24 | { 25 | "model": "gpt-4o", 26 | "api_key": userdata.get('openai_key') 27 | } 28 | ] 29 | } 30 | ``` 31 | 32 | ## Task 2: Create simple Two-Agent Chat 33 | 34 | 1. First, let's create an assistant agent. This type of agent doesn't require human input and it cannot execute code. However it can generate the code. 35 | 36 | ```python 37 | import os 38 | from autogen import AssistantAgent, UserProxyAgent 39 | 40 | assistant = AssistantAgent(name="assistant", llm_config=config) 41 | ``` 42 | 43 | 2. AssistantAgent comes with default system message. You can examine it: 44 | 45 | ```python 46 | print(assistant.system_message) 47 | ``` 48 | 49 | 3. Human Input Mode is set to **NEVER**. That means that this agent will never ask for user input. Verify this: 50 | 51 | ```python 52 | print(assistant.human_input_mode) 53 | ``` 54 | 55 | 4. Now, let's create a code executor. In AutoGen we can execute the code: 56 | - locally 57 | - in Docker (safest way) 58 | - in Jupyter 59 | For now let's use local mode: 60 | 61 | ```python 62 | from autogen.coding import LocalCommandLineCodeExecutor 63 | 64 | code_executor = LocalCommandLineCodeExecutor() 65 | ``` 66 | 67 | 5. Then create a UserProxyAgent. This type of agent doesn't use LLM but it can execute the code. By default it asks user before execution: 68 | 69 | ```python 70 | user_proxy = UserProxyAgent( 71 | name="user_proxy", 72 | code_execution_config={"executor": code_executor} 73 | ) 74 | ``` 75 | 76 | 6. Because this agent doesn't use LLM, the system message is empty: 77 | 78 | ```python 79 | print(user_proxy.system_message) 80 | ``` 81 | 82 | 7. Now let's give a task to our agents: 83 | 84 | ```python 85 | user_proxy.initiate_chat( 86 | assistant, 87 | message="""What is the current bitcoin price? Draw a chart of bitcoin price from last 6 month and save to file.""", 88 | ) 89 | ``` 90 | 91 | 8. Whenever to **user_proxy** agent is called, you will need to hit **Enter** to continue. But in the end you should get two new files. One with the chart and second with python code. 92 | 93 | 9. You can modify this behaviour and make those agents fully autonomous: 94 | 95 | ```python 96 | user_proxy = UserProxyAgent( 97 | name="user_proxy", 98 | code_execution_config={"executor": code_executor}, 99 | human_input_mode="NEVER" 100 | ) 101 | ``` 102 | 103 | 10. Run the chat again: 104 | 105 | ```python 106 | user_proxy.initiate_chat( 107 | assistant, 108 | message="""What is the current bitcoin price? Draw a chart of bitcoin price from last 6 month and save to file.""", 109 | ) 110 | ``` 111 | 112 | ## Task 3: Create a group of agents 113 | In this task a group of agents will work together to write a blog post. 114 | 115 | 1. First, let's create a planner, that will plan how to blog post should be structured: 116 | 117 | ```python 118 | planner = AssistantAgent( 119 | name="Planner", 120 | description="Plans the whole work and split it into sub-tasks", 121 | llm_config=config, 122 | system_message="""You are a helpful AI assistant. Your role is to create tasks. 123 | You suggest coding and reasoning steps for another AI assistant to accomplish a task. 124 | Do not suggest concrete code. For any action beyond writing code or reasoning, convert it to a step that can be implemented by writing code. 125 | For example, browsing the web can be implemented by writing code that reads and prints the content of a web page. 126 | Finally, inspect the execution result. If the plan is not good, suggest a better plan. 127 | If the execution is wrong, analyze the error and suggest a fix. When the plan is done - TERMINATE""", 128 | is_termination_msg=lambda msg: "TERMINATE" in msg["content"].lower() 129 | ) 130 | ``` 131 | 132 | 2. Writer: 133 | 134 | ```python 135 | writer = AssistantAgent( 136 | name="Writer", 137 | description="Writes content", 138 | llm_config=config, 139 | system_message="""Content writer. You follow an approved plan. You write a blog post based 140 | on a given plan and topic. You should include code snippets. Write the blog post in a separate file. 141 | """, 142 | ) 143 | ``` 144 | 145 | 3. Critic: 146 | 147 | ```python 148 | critic = AssistantAgent( 149 | name="Critic", 150 | description="Checks and analyze text", 151 | llm_config=config, 152 | system_message="""Critic. You are a helpful AI assistant. Your role is to check text generated by Writer 153 | and report mistakes and places for improvemement. When the blog post is ok - TERMINATE. 154 | """, 155 | is_termination_msg=lambda msg: "TERMINATE" in msg["content"].lower() 156 | ) 157 | ``` 158 | 159 | 4. Executor: 160 | 161 | ```python 162 | executor = UserProxyAgent( 163 | name="Executor", 164 | description="Execudes Python code", 165 | human_input_mode="NEVER", 166 | code_execution_config={"executor": code_executor} 167 | ) 168 | ``` 169 | 170 | 5. User Proxy: 171 | 172 | ```python 173 | user_admin = UserProxyAgent( 174 | name="Admin", 175 | system_message="A human admin.", 176 | code_execution_config=False, 177 | human_input_mode="TERMINATE" 178 | ) 179 | ``` 180 | 181 | 6. Now, let's create a group chat and a manager. They are responsible for managing the whole conversation. We limit the conversation to 25 messages. 182 | By default AutoGen automatically selects which agent should talk next. It uses mainly **description** attribute of each agent. Other options of an order are: **round robin**, **random**, **manual**. 183 | 184 | ```python 185 | from autogen import GroupChat, GroupChatManager 186 | 187 | groupchat = GroupChat( 188 | agents=[user_admin, planner, writer, critic, executor], messages=[], max_round=25 189 | ) 190 | manager = GroupChatManager(groupchat=groupchat, llm_config=config) 191 | ``` 192 | 193 | 7. Run the group: 194 | 195 | ```python 196 | user_admin.initiate_chat( 197 | manager, 198 | message=""" 199 | Write a blog post about a GPT function calling. The post should contain one example. 200 | """ 201 | ) 202 | ``` 203 | 204 | 8. Try to ask different question! 205 | 206 | ## End Lab 207 | -------------------------------------------------------------------------------- /03_AI_agents/Lab1/README.md: -------------------------------------------------------------------------------- 1 | # Lab 1: Building AI agent 2 | In this lab you will learn how to build an AI agent and how to configure it. 3 | 4 | ## Prerequisutes: 5 | - OpenAI API key with a few Euros credits 6 | - Google account 7 | 8 | ## Task 1: Set-up 9 | 1. Open Google Colab: https://colab.research.google.com/ 10 | 2. Create new notebook, name it eg. **Workshop3 - la1** 11 | 3. First, we need to install dependencies. In the first cell type and run: 12 | 13 | ```python 14 | !pip install --quiet langchain==0.2.16 langchain-openai==0.1.23 langchain-community==0.2.16 qdrant-client==1.12.0 langchainhub==0.1.21 tavily-python==0.5.0 15 | ``` 16 | 17 | 4. Connecto to LansSmith for debuggin purposes: 18 | 19 | ```python 20 | import os 21 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 22 | os.environ["LANGCHAIN_PROJECT"] = "ai-workshops-day3" 23 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 24 | os.environ["LANGCHAIN_API_KEY"] = "" 25 | ``` 26 | 27 | 5. Go to https://tavily.com/ and create a free account there. Tavily is a search engine API, it integrates well with Langchain. 28 | 29 | 6. Grab the API key from Tavily and create new secret in Colab named **tavily_key**. Paste your API key there. 30 | 31 | 7. Finally, let's set-up keys and initiate the model: 32 | 33 | ```python 34 | from google.colab import userdata 35 | from langchain_openai import ChatOpenAI 36 | 37 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 38 | gpt4 = ChatOpenAI(model = "gpt-4o", temperature = 0) 39 | ``` 40 | 41 | ## Task 2: Create the agent 42 | 1. First, create a Tavily tool and a simple list of tools with just one element. 43 | 44 | ```python 45 | from langchain_community.tools.tavily_search import TavilySearchResults 46 | 47 | os.environ["TAVILY_API_KEY"] = userdata.get('tavily_key') 48 | 49 | tavily_tool = TavilySearchResults() 50 | tools = [tavily_tool] 51 | ``` 52 | 53 | 2. You can invoke the tool and see how it works: 54 | 55 | ```python 56 | tavily_tool.invoke("funny cats") 57 | ``` 58 | 59 | 3. Let's add agent's prompt. Note the {agent_scratchpad} placeholder for agent's notes 60 | 61 | ```python 62 | from langchain.agents import AgentExecutor, create_tool_calling_agent 63 | from langchain_core.prompts import ( 64 | ChatPromptTemplate, 65 | MessagesPlaceholder, 66 | ) 67 | 68 | prompt = ChatPromptTemplate.from_messages( 69 | [ 70 | ("system", "You are a helpful assistant. Use all the tools to answer the question"), 71 | ("human", "{input}"), 72 | ("placeholder", "{agent_scratchpad}"), 73 | ] 74 | ) 75 | ``` 76 | 77 | 4. Now let's create the agent. We are using here **Tool Calling** agent which requires a LLM with tools support. Most of new models support it (OpenAI, Anthropic, Gemini, Mistral etc.). We also create an instance of AgentExecutor class which is responsible for running the agent. 78 | 79 | ```python 80 | agent = create_tool_calling_agent(gpt4, tools, prompt) 81 | agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True) 82 | ``` 83 | 84 | 5. Ok, let's invoke it: 85 | 86 | ```python 87 | agent_executor.invoke({"input": "Who is the coach of polish voleyball teams and what is his age?"}) 88 | ``` 89 | 90 | 6. Try something more sophisticated: 91 | 92 | ```python 93 | agent_executor.invoke({"input": "Which teams played in the last final of man's voleyball world cup? What was the squad of each team? Give me also a nationality of each coach"}) 94 | ``` 95 | 96 | 7. Or give some hints: 97 | 98 | ```python 99 | agent_executor.invoke({"input": "Which teams played in the last final of man's voleyball world cup? What was the squad of each team? Give me also a nationality of each coach. When responding first check the year of last world cup "}) 100 | ``` 101 | 102 | ## Task 3: Add history to the agent 103 | 1. Let's modify the prompt: 104 | 105 | ```python 106 | history_prompt = ChatPromptTemplate.from_messages( 107 | [ 108 | ( 109 | "system", 110 | """You are a helpful assistant. Use all the tools to answer the question""", 111 | ), 112 | MessagesPlaceholder(variable_name="chat_history", optional=True), 113 | ("human", "{input}"), 114 | MessagesPlaceholder(variable_name="agent_scratchpad", optional=True), 115 | ] 116 | ) 117 | ``` 118 | 119 | 2. And a memory. We will use simple in-memory **ChatMessageHistory** class. Note, that "chat_history" key needs to match the placeholder name in a prompt. 120 | 121 | ```python 122 | from langchain.memory import ConversationBufferMemory 123 | from langchain.memory import ChatMessageHistory 124 | 125 | chat_history = ChatMessageHistory() 126 | 127 | memory = ConversationBufferMemory( 128 | chat_memory=chat_history, 129 | memory_key='chat_history', 130 | return_messages=True, 131 | input_key='input', 132 | output_key='output' 133 | ) 134 | ``` 135 | 136 | 3. Let's create the agent again with memory now: 137 | 138 | ```python 139 | agent = create_tool_calling_agent(gpt4, tools, history_prompt) 140 | agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True) 141 | ``` 142 | 143 | 4. And test it: 144 | ```python 145 | agent_executor.invoke({"input": "Which teams won the last final of man's voleyball world cup? "}) 146 | 147 | agent_executor.invoke({"input": "Give me a squad of Polish team"}) 148 | 149 | agent_executor.invoke({"input": "What was my last question?"}) 150 | ``` 151 | 152 | 5. We can also add flag that will show us how agent was "thinking": 153 | 154 | ```python 155 | agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True, return_intermediate_steps=True) 156 | agent_executor.invoke({"input": "Search for information about wombats"}) 157 | ``` 158 | 159 | ## Task 4: Use ReAct agent 160 | 1. Now let's use ReAct agent that doesn't use tool calling but [ReAct prompting](https://www.promptingguide.ai/techniques/react) instead. 161 | 162 | 2. We will use Langchain hub here. It is a public repository with prompts. Go to: https://smith.langchain.com/hub and look for "hwchase17/react". Inspect how this prompt is build. 163 | 164 | 3. Let's use this prompt in a code: 165 | 166 | ```python 167 | from langchain import hub 168 | from langchain.agents import create_react_agent 169 | 170 | prompt = hub.pull("hwchase17/react") 171 | ``` 172 | 173 | 4. And let's create the agent and agent executor: 174 | 175 | ```python 176 | agent = create_react_agent(gpt4, tools, prompt) 177 | agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True) 178 | ``` 179 | 180 | 5. See the agent in action: 181 | 182 | ```python 183 | agent_executor.invoke({"input": "Which teams played in the last final of man's voleyball world cup? What was the squad of each team? Give me also a nationality of each coach. When responding first check the year of last world cup "}) 184 | ``` 185 | 186 | ## End lab -------------------------------------------------------------------------------- /03_AI_agents/Lab2/README.md: -------------------------------------------------------------------------------- 1 | # Lab 2: Building multi-tools agent 2 | In this lab you will learn how to build an AI agent with multiple tools. 3 | 4 | ## Prerequisutes: 5 | - OpenAI API key with a few Euros credits 6 | - Google account 7 | - Tavily API key 8 | - Qdrant vector database with populated data (optionally) [see: Day2] 9 | 10 | ## Task 1: Set-up 11 | 1. Open Google Colab: https://colab.research.google.com/ 12 | 2. Create new notebook, name it eg. **Workshop3 - lab2** 13 | 3. First, we need to install dependencies. In the first cell type and run: 14 | 15 | ```python 16 | !pip install --quiet \ 17 | langchain==0.2.16 \ 18 | langchain-openai==0.1.23 \ 19 | qdrant-client==1.12.0 \ 20 | langchainhub==0.1.21 \ 21 | langchain-community==0.2.16 \ 22 | tavily-python==0.5.0 \ 23 | langchain-experimental==0.0.65 \ 24 | SQLAlchemy==2.0.36 25 | ``` 26 | 27 | 4. Connecto to LansSmith for debuggin purposes: 28 | 29 | ```python 30 | import os 31 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 32 | os.environ["LANGCHAIN_PROJECT"] = "ai-workshops-day3" 33 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 34 | os.environ["LANGCHAIN_API_KEY"] = "" 35 | ``` 36 | 37 | 5. Finally, let's set-up keys and initiate the model. This time we will use gpt-4-turbo: 38 | 39 | ```python 40 | from google.colab import userdata 41 | from langchain_openai import ChatOpenAI 42 | 43 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 44 | gpt4 = ChatOpenAI(model = "gpt-4-turbo", temperature = 0) 45 | ``` 46 | 47 | ## Task 2: Set-up SQL database 48 | We will use data from SQL database with SQLite engine. 49 | 50 | 1. Download **Chinook.db** file from repository and upload it into files on Colab. 51 | 2. Activate SQL extension in Colab. In a new cell paste: 52 | ```sql 53 | %load_ext sql 54 | ``` 55 | 3. Load the database: 56 | ```sql 57 | %%sql 58 | sqlite:///Chinook.db 59 | ``` 60 | 4. Now you should be able to list all tables in database: 61 | ```sql 62 | %%sql 63 | SELECT name FROM sqlite_master 64 | WHERE type='table'; 65 | ``` 66 | 5. And execute simple query: 67 | ```sql 68 | %%sql 69 | select BillingCountry, sum(Total) from Invoice group by BillingCountry; 70 | ``` 71 | 72 | ## Task 3: Set-up all tools 73 | 74 | 1. Set-up Tavily tool: 75 | ```python 76 | from langchain_community.tools.tavily_search import TavilySearchResults 77 | 78 | os.environ["TAVILY_API_KEY"] = userdata.get('tavily_key') 79 | 80 | tavily_tool = TavilySearchResults() 81 | ``` 82 | 83 | 2. **You can omit this if you haven't completed Lab2 from Day2**. Here we will configure a retriever tool. It will retrieve data from our vector database. 84 | >Notice that create_retriever_tool() method needs a tool name and description. It is important because LLM bases on these descriptions. 85 | 86 | ```python 87 | from qdrant_client import QdrantClient 88 | from langchain_openai import OpenAIEmbeddings 89 | from langchain_community.vectorstores import Qdrant 90 | from langchain.tools.retriever import create_retriever_tool 91 | 92 | collection_name = "labor_law" 93 | embeddings = OpenAIEmbeddings(model = "text-embedding-3-large") 94 | 95 | qdrant_client = QdrantClient( 96 | url="", 97 | api_key=userdata.get('qdrant_key') 98 | ) 99 | qdrant = Qdrant(qdrant_client, collection_name, embeddings) 100 | 101 | retriever_tool = create_retriever_tool( 102 | qdrant.as_retriever(), 103 | "search_documents", 104 | "Searches and returns documents about labor law in Poland", 105 | ) 106 | ``` 107 | 108 | 3. Remember to put your **Qdrant URL** above and **qdrant_key** into secrets. 109 | 110 | 4. Now, let's create a PythonREPL tool that can write and execute Python code locally: 111 | 112 | ```python 113 | from langchain_experimental.utilities import PythonREPL 114 | from langchain_core.tools import Tool 115 | 116 | python_repl = PythonREPL() 117 | 118 | repl_tool = Tool( 119 | name="python_repl", 120 | description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.", 121 | func=python_repl.run, 122 | ) 123 | ``` 124 | 125 | 5. Create SQLDatabase toolkit. Toolkit is a set of tools. Here we load all tools that can interact with a database: 126 | 127 | ```python 128 | from langchain_community.utilities.sql_database import SQLDatabase 129 | from langchain_community.agent_toolkits import SQLDatabaseToolkit 130 | 131 | db = SQLDatabase.from_uri("sqlite:///Chinook.db") 132 | 133 | toolkit = SQLDatabaseToolkit(db=db, llm=gpt4) 134 | context = toolkit.get_context() 135 | db_tools = toolkit.get_tools() 136 | ``` 137 | 138 | 6. Inspect what tools are inside: 139 | 140 | ```python 141 | db_tools 142 | ``` 143 | 144 | 7. Now, let's combine all tools into signle list: 145 | 146 | ```python 147 | all_tools = [] 148 | all_tools += db_tools 149 | all_tools.append(retriever_tool) 150 | all_tools.append(tavily_tool) 151 | all_tools.append(repl_tool) 152 | 153 | all_tools 154 | ``` 155 | 156 | ## Task 4: Multi-tools agent 157 | 1. Create the prompt and the agent: 158 | 159 | ```python 160 | from langchain.agents import AgentExecutor, create_tool_calling_agent 161 | from langchain_core.prompts import ( 162 | ChatPromptTemplate, 163 | MessagesPlaceholder, 164 | ) 165 | 166 | prompt = ChatPromptTemplate.from_messages( 167 | [ 168 | ("system", "You are a helpful assistant. Use all the tools to answer the question"), 169 | ("human", "{input}"), 170 | ("placeholder", "{agent_scratchpad}"), 171 | ] 172 | ) 173 | prompt = prompt.partial(**context) 174 | 175 | agent = create_tool_calling_agent(gpt4, all_tools, prompt) 176 | agent_executor = AgentExecutor(agent=agent, tools=all_tools, verbose=True) 177 | ``` 178 | 179 | 2. Test it: 180 | ```python 181 | agent_executor.invoke( 182 | { 183 | "input": "What is the 10th fibonacci number?" 184 | } 185 | ) 186 | ``` 187 | ```python 188 | agent_executor.invoke( 189 | { 190 | "input": "Give me list of invoice sales amount per each country" 191 | } 192 | ) 193 | ``` 194 | ```python 195 | agent_executor.invoke( 196 | { 197 | "input": "Give me list of invoice sales amount per each country. Then draw a barchart" 198 | } 199 | ) 200 | ``` 201 | ```python 202 | agent_executor.invoke( 203 | { 204 | "input": """Give me list of invoice sales amount per each country. Then draw a barchart. Then take country with 205 | highest value and search the web for the amount of days of holiday leave in this country. 206 | """ 207 | } 208 | ) 209 | ``` 210 | ```python 211 | agent_executor.invoke( 212 | { 213 | "input": """Give me list of invoice sales amount per each country. Then draw a barchart. Then take country with 214 | highest value and search the web for the amount PTO in this country. 215 | Finally, compare the PTO with the holiday leave days in Poland. 216 | """ 217 | } 218 | ) 219 | ``` 220 | 221 | 3. You can also try to insert some data: 222 | 223 | ```python 224 | agent_executor.invoke( 225 | { 226 | "input": "Insert new Artist named Zenek Martyniuk" 227 | } 228 | ) 229 | ``` -------------------------------------------------------------------------------- /02_Vector database & RAG/Lab2/README.md: -------------------------------------------------------------------------------- 1 | # Lab 2: Building RAG 2 | In this lab we will build a RAG that works with our knowledge base. 3 | 4 | ## Prerequisutes: 5 | - OpenAI API key with a few Euros credits 6 | - Google account 7 | - Lab1 finished 8 | 9 | ## Task 1: Set-up 10 | 1. Open Google Colab: https://colab.research.google.com/ 11 | 2. Create new notebook, name it eg. **Workshop2 - la2** 12 | 3. First, we need to install dependencies. In the first cell type and run: 13 | 14 | ```python 15 | !pip install --quiet langchain==0.2.16 langchain-openai==0.1.23 langchain-community==0.2.16 qdrant-client==1.12.0 16 | ``` 17 | 18 | 4. Connecto to LansSmith for debuggin purposes: 19 | 20 | ```python 21 | import os 22 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 23 | os.environ["LANGCHAIN_PROJECT"] = "ai-workshops-day2" 24 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 25 | os.environ["LANGCHAIN_API_KEY"] = "" 26 | ``` 27 | 28 | 5. Configure Qdrant client, embeddings model and LLM: 29 | 30 | ```python 31 | import os 32 | from qdrant_client import QdrantClient 33 | from langchain_openai import OpenAIEmbeddings 34 | from google.colab import userdata 35 | from langchain_community.vectorstores import Qdrant 36 | from langchain_openai import ChatOpenAI 37 | 38 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 39 | 40 | embeddings = OpenAIEmbeddings(model = "text-embedding-3-large") 41 | collection_name = "labor_law" 42 | 43 | qdrant_client = QdrantClient( 44 | url="", 45 | api_key=userdata.get('qdrant_key') 46 | ) 47 | qdrant = Qdrant(qdrant_client, collection_name, embeddings) 48 | 49 | gpt4 = ChatOpenAI(model = "gpt-4o") 50 | ``` 51 | 52 | ## Task 2: Create RAG chain 53 | 1. First, we need to create a retreiver. It is a Langchain object that allows to retrieve content from data sources: 54 | 55 | ```python 56 | retriever = qdrant.as_retriever() 57 | ``` 58 | Alternatively, you can also use MMR: 59 | 60 | ```python 61 | retriever = qdrant.as_retriever(search_type="mmr") 62 | ``` 63 | 64 | 2. Now let's create a prompt and a simple function that concatenates multiple documents into one: 65 | 66 | ```python 67 | from langchain_openai import ChatOpenAI 68 | from langchain_core.prompts import PromptTemplate 69 | 70 | prompt = PromptTemplate.from_template("""Use the following pieces of context to answer the question at the end. 71 | If you don't know the answer, just say that you don't know, don't try to make up an answer. 72 | Use three sentences maximum and keep the answer as concise as possible. 73 | Always say "czy mogę jeszcze jakoś pomóc?" at the end of the answer. 74 | 75 | {context} 76 | 77 | Question: {question} 78 | 79 | Helpful Answer:""") 80 | 81 | def format_docs(docs): 82 | return "\n\n".join(doc.page_content for doc in docs) 83 | ``` 84 | 85 | 3. Let's construct the RAG chain: 86 | 87 | ```python 88 | from langchain_core.output_parsers import StrOutputParser 89 | from langchain_core.runnables import RunnablePassthrough 90 | 91 | rag = ( 92 | { 93 | "context": retriever | format_docs, 94 | "question": RunnablePassthrough(), 95 | } 96 | | prompt 97 | | gpt4 98 | | StrOutputParser() 99 | ) 100 | ``` 101 | 102 | 4. And invoke: 103 | 104 | ```python 105 | rag.invoke("Ile przysługuje dni urlopu wypoczynkowego?") 106 | ``` 107 | 108 | 5. Try to ask different questions :) 109 | 110 | ## Task 3: Add history to the chat 111 | In order to create chatbot with history we can use prebuild chains. 112 | 113 | 1. First, let's create a **history_aware_retriever**. This retriever summarizes the history and reformulates the question if needed: 114 | 115 | ```python 116 | from langchain.chains import create_history_aware_retriever 117 | from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder 118 | 119 | contextualize_system_prompt = """Given a chat history and the latest user question \ 120 | which might reference context in the chat history, formulate a standalone question \ 121 | which can be understood without the chat history. Do NOT answer the question, \ 122 | just reformulate it if needed and otherwise return it as is.""" 123 | contextualize_prompt = ChatPromptTemplate.from_messages( 124 | [ 125 | ("system", contextualize_system_prompt), 126 | MessagesPlaceholder("chat_history"), 127 | ("human", "{input}"), 128 | ] 129 | ) 130 | history_aware_retriever = create_history_aware_retriever( 131 | gpt4, retriever, contextualize_prompt 132 | ) 133 | ``` 134 | 135 | 2. Then, we need to create a **stuff_document_chain** that can pass a list of documents to the model: 136 | 137 | ```python 138 | from langchain.chains import create_retrieval_chain 139 | from langchain.chains.combine_documents import create_stuff_documents_chain 140 | 141 | qa_system_prompt = """You are an assistant for question-answering tasks. \ 142 | Use the following pieces of retrieved context to answer the question. \ 143 | If you don't know the answer, just say that you don't know. \ 144 | Use three sentences maximum and keep the answer concise.\ 145 | 146 | {context}""" 147 | qa_prompt = ChatPromptTemplate.from_messages( 148 | [ 149 | ("system", qa_system_prompt), 150 | MessagesPlaceholder("chat_history"), 151 | ("human", "{input}"), 152 | ] 153 | ) 154 | 155 | 156 | question_answer_chain = create_stuff_documents_chain(gpt4, qa_prompt) 157 | ``` 158 | 159 | 3. Now, let's create **retrieval_chain**. It takes as the arguments the retriever and stuff_documents chain 160 | 161 | ```python 162 | rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain) 163 | ``` 164 | 165 | 4. Now, it's time to create history storage. For this example we will use simple **ChatMessageHistory** that just stores chat history in memory. 166 | >Langchain supports many different message history providers. You can check them [here](https://python.langchain.com/v0.1/docs/integrations/memory/) 167 | 168 | ```python 169 | from langchain_core.chat_history import BaseChatMessageHistory 170 | from langchain_community.chat_message_histories import ChatMessageHistory 171 | from langchain_core.runnables.history import RunnableWithMessageHistory 172 | 173 | store = {} 174 | 175 | def get_session_history(session_id: str) -> BaseChatMessageHistory: 176 | if session_id not in store: 177 | store[session_id] = ChatMessageHistory() 178 | return store[session_id] 179 | ``` 180 | It simply just creates a dict object with sessions and corresponding history objects 181 | 182 | 5. So, let's create a final chain: 183 | 184 | ```python 185 | conversational_rag_chain = RunnableWithMessageHistory( 186 | rag_chain, 187 | get_session_history, 188 | input_messages_key="input", 189 | history_messages_key="chat_history", 190 | output_messages_key="answer", 191 | ) 192 | ``` 193 | 194 | 6. And run it: 195 | 196 | ```python 197 | conversational_rag_chain.invoke( 198 | {"input": "Ile przysługuje dni urlopu wypoczynkowego?"}, 199 | config={ 200 | "configurable": {"session_id": "123"} 201 | }, # constructs a key "123" in `store`. 202 | ) 203 | ``` 204 | 205 | 7. Try different questions, for example: 206 | - O co pytałem poprzednio? 207 | - Jak aplikować o urlop maciezyński? 208 | 209 | Try also changing the session id. 210 | 211 | 8. Check the memory content: 212 | ```python 213 | display(store) 214 | ``` 215 | 216 | 9. Investigate the Langsmith monitoring. 217 | 218 | ## End lab -------------------------------------------------------------------------------- /01_Langchain basics/Lab2/README.md: -------------------------------------------------------------------------------- 1 | # Lab 2: LinkedIn post generator 2 | In this lab you will build a simple LinkedIn post generator. 3 | 4 | ## Prerequisutes: 5 | - OpenAI API key with a few Euros credits 6 | - Google account 7 | 8 | ## Task 1: Set-up 9 | 1. Open Google Colab: https://colab.research.google.com/ 10 | 1. Create new notebook, name it eg. **Workshop1 - lab2** 11 | 1. First, we need to install dependencies. In the first cell type and run: 12 | 13 | ```python 14 | !pip install --quiet langchain==0.2.16 langchain-openai==0.1.23 langchain-community==0.2.16 15 | ``` 16 | 17 | Here we install Langchain framework and langchain-openai responsible for OpenAI integration. 18 | 1. In the next cell create an instance of gpt-4o-mini model: 19 | 20 | ```python 21 | import os 22 | from langchain_openai import ChatOpenAI 23 | from langchain_core.prompts import ChatPromptTemplate 24 | from langchain_core.output_parsers import StrOutputParser 25 | from langchain_core.runnables import RunnableLambda 26 | from langchain_community.utilities.dalle_image_generator import DallEAPIWrapper 27 | from langchain_core.runnables import RunnableParallel, RunnablePassthrough 28 | from langchain.output_parsers.openai_tools import PydanticToolsParser 29 | from langchain_core.pydantic_v1 import BaseModel, Field 30 | from google.colab import userdata 31 | 32 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 33 | 34 | gpt4 = ChatOpenAI(model = "gpt-4o-mini") 35 | ``` 36 | 37 | 2. Switch on LangSmith: 38 | 39 | ```python 40 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 41 | os.environ["LANGCHAIN_PROJECT"] = "ai-agent-workshops" 42 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 43 | os.environ["LANGCHAIN_API_KEY"] = "YOUR_LANGSMITH_KEY" 44 | ``` 45 | 46 | ## Task 2: LinkedIn post generator! 47 | Now let's combine all the knowledge and let's build a simple LinkedIn post generator. :) 48 | Here is the idea: 49 | - first, generate post content 50 | - then, do text correction, remove errors, add emojis 51 | - generate also image for the post and hashtags 52 | So the chain will looks like this: generate post content -> correct the post -> generate image -> generate hashtags 53 | 54 | 1. First, generating the post content: 55 | 56 | ```python 57 | output_parser = StrOutputParser() 58 | 59 | generate_content_prompt = ChatPromptTemplate.from_template("Based the outline: {outline} generate me a LinkedIn post. Use AIDA model. Do not generate hashtags. Use {language} language") 60 | 61 | generate_content_chain = generate_content_prompt | gpt4 | output_parser 62 | ``` 63 | 64 | 2. Try how it works: 65 | 66 | ```python 67 | outline = """ 68 | - I've attended the AI workshop 69 | - I've created my first app using Langchain 70 | - I've built my own chatbot 71 | - I've created a LinkedIn post generator 72 | - I recommend this workshops 73 | mark @Piotr Kalinowski 74 | """ 75 | post = generate_content_chain.invoke({"outline": outline, "language": "polish"}) 76 | print(post) 77 | ``` 78 | 79 | 3. Now the correction prompt and dalle metaprompt: 80 | 81 | ```python 82 | correction_prompt = ChatPromptTemplate.from_template("Correct this LinkedIn post: {post}. Remove errors, unnecessary interjections and AIDA expressions. Add emojis. Do not change language") 83 | 84 | dalle_metaprompt = ChatPromptTemplate.from_template("Generate a detailed prompt to generate a photograph for the LinkedIn post. Here is the post text: '{post}'. Use maximum 3 sentences") 85 | ``` 86 | 87 | 4. In order to generate image we need to execute **DallEAPIWrapper().run()** method. However **DallEAPIWrapper** does not implements **Runnable** interface. That is why we need to wrap it in a function and use **RunnableLambda** that transform a function into **Runnable**: 88 | 89 | ```python 90 | def run_dalle(prompt): 91 | return DallEAPIWrapper(model='dall-e-3').run(prompt + " High resolution, 2K, 4K, 8K, clear, good lighting, detailed, extremely detailed, sharp focus, intricate, beautiful, realistic+++, complementary colors, high quality, hyper detailed, masterpiece, best quality, artstation, stunning") 92 | ``` 93 | (Notice: We are adding to the prompt some image prompt engineering) 94 | 95 | 5. In order to generate hastags list we will use a tool: 96 | 97 | ```python 98 | class hashtag_generator(BaseModel): 99 | """Generates hashtags for a LinkedIn post.""" 100 | 101 | hastags: list[str] = Field(..., description="A list of LinkedIn hashtags. Focus mostly on AI") 102 | 103 | tools = [hashtag_generator] 104 | ``` 105 | 6. Ok, let's combine everything together: 106 | 107 | ```python 108 | chain = ( 109 | generate_content_chain 110 | | (lambda input: {"post": input}) 111 | | correction_prompt 112 | | gpt4 113 | | output_parser 114 | | gpt4.bind_tools(tools) 115 | ) 116 | chain.invoke({"outline": outline, "language": "polish"}) 117 | ``` 118 | It works, however it is not what we are looking for. As a result we want to have a post content, list of hashtags and an image. 119 | 120 | 7. Let's use a **RunnableParallel** class. It can execute multiple chains in parallel. First let's define two chains. One for dalle and second for hashtags: 121 | 122 | ```python 123 | dalle_chain = dalle_metaprompt | gpt4 | output_parser | RunnableLambda(run_dalle) 124 | 125 | def hashtags_to_string(hashtag_generator): 126 | result = "" 127 | for tag in hashtag_generator[0].hastags: 128 | result += "#" + tag + " " 129 | return result 130 | 131 | hashtags_chain = gpt4.bind_tools(tools) | PydanticToolsParser(tools=tools) | RunnableLambda(hashtags_to_string) 132 | ``` 133 | For the hashtags we used PydanticToolsParser that simple parses the tools output. It facilities usage of **hashtags_to_string** function 134 | 135 | 8. And finally! Here is our last chain: 136 | 137 | ```python 138 | chain = ( 139 | generate_content_chain 140 | | (lambda input: {"post": input}) 141 | | correction_prompt 142 | | gpt4 143 | | output_parser 144 | | RunnableParallel(post = RunnablePassthrough() ,hashtags = hashtags_chain, image = dalle_chain) 145 | ) 146 | ``` 147 | We've also used here **RunnablePassthrough** class that just passes the input into the output unchanged. This typically is used in conjuction with RunnableParallel to pass data to a new key in the map. 148 | 9. Create helper function: 149 | 150 | ```python 151 | def display_image(image_url): 152 | try: 153 | import google.colab 154 | IN_COLAB = True 155 | except ImportError: 156 | IN_COLAB = False 157 | 158 | if IN_COLAB: 159 | from google.colab.patches import cv2_imshow # for image display 160 | from skimage import io 161 | import cv2 162 | 163 | image = io.imread(image_url) 164 | image_bgr = cv2.cvtColor(image, cv2.COLOR_RGB2BGR) # Convert RGB to BGR 165 | cv2_imshow(image_bgr) 166 | else: 167 | import cv2 168 | from skimage import io 169 | 170 | image = io.imread(image_url) 171 | image_bgr = cv2.cvtColor(image, cv2.COLOR_RGB2BGR) # Convert RGB to BGR 172 | cv2.imshow("image", image_bgr) 173 | cv2.waitKey(0) # wait for a keyboard input 174 | cv2.destroyAllWindows() 175 | ``` 176 | 177 | 10. Invoke the generator! 178 | 179 | ```python 180 | result = chain.invoke({"outline": outline, "language": "polish"}) 181 | print(result["post"]) 182 | print(result["hashtags"]) 183 | ``` 184 | 185 | 11. And display post image: 186 | ```python 187 | display_image(result["image"]) 188 | ``` 189 | 190 | 11. Investigate the LangSmith! 191 | 12. Put the post on your LinkedIn ;) 192 | 193 | ## End lab 194 | 195 | ### Final questions and concerns: 196 | - How many token were used? 197 | - how much money this lab consumed :) -------------------------------------------------------------------------------- /02_Vector database & RAG/Lab1/README.md: -------------------------------------------------------------------------------- 1 | # Lab 1: Data embedding and vectorization 2 | In this lab we will create a knowledge base for a labor law data. You will learn: 3 | - Basics of embedding 4 | - How to load documents into vector database 5 | - How to search for documents in vector database 6 | 7 | ## Prerequisutes: 8 | - OpenAI API key with a few Euros credits 9 | - Google account 10 | 11 | ## Task 1: Create vector database 12 | For the vector database we will use Qdrant in a free cloud-based version. 13 | 1. Go to https://cloud.qdrant.io/ and create a new account. 14 | 2. When logged in, click **Clusters** on the left menu and create a new FREE cluster 15 | 3. Grab tha API key and cluster url 16 | 4. When the cluster is created, select it and click on **Open Dashboard** button. 17 | 18 | 19 | ## Task 2: Set-up 20 | 1. Open Google Colab: https://colab.research.google.com/ 21 | 2. Create new notebook, name it eg. **Workshop2 - lab1** 22 | 3. First, we need to install dependencies. In the first cell type and run: 23 | 24 | ```python 25 | !pip install --quiet langchain==0.2.16 langchain-openai==0.1.23 langchain-community==0.2.16 qdrant-client==1.12.0 pypdf==5.0.1 26 | ``` 27 | 28 | Here we install: 29 | - Langchain framework 30 | - langchain-openai responsible for OpenAI integration. 31 | - qdrant-client responsible for Qdrant integration. 32 | - pypdf responsible for PDF parsing. 33 | 34 | 4. In the left menu of Google Colab click **KEY** button. It is a place when you can store your secrets. 35 | 5. Create two new secrets: 36 | - openai_key 37 | - qdrant_key 38 | 39 | Make both of them accessible from notebooks. 40 | 41 | 5. In the next cell paste your OpenAI API key and create an instance of the **text-embedding-3-large** model: 42 | 43 | ```python 44 | import os 45 | from langchain_openai import OpenAIEmbeddings 46 | from google.colab import userdata 47 | 48 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 49 | 50 | embeddings = OpenAIEmbeddings(model = "text-embedding-3-large") 51 | ``` 52 | 53 | 6. In this lab we will index documents about labor law in Poland. Download these two files: 54 | - kodeks_pracy.pdf 55 | - praca_rodzicielstwo.pdf 56 | 57 | (they can be found on the GitHub repo Day2/Lab1) 58 | 59 | In **Google Colab** click **Files** button on the left menu and create **docs** directory. Upload both files there. 60 | 61 | ## Task 3: Load documents into vector database 62 | 1. First we need to load our documents from file system. We will use **DirectoryLoader** for that. 63 | 64 | >Langchain supports loading data from many different sources (eg. Google Drive, Sharepoint, Web, Wikipedia and much, much more). You can check all options in [the docs](https://python.langchain.com/v0.1/docs/integrations/document_loaders/). 65 | 66 | > DirectoryLoader uses **Unstructured** by default to load documents from a PDF file. This lib is flexible, in can also load eg. docx, pptx, but it is also large. You can change the loader to load class if needed. 67 | 68 | ```python 69 | from langchain_community.document_loaders import DirectoryLoader 70 | from langchain_community.document_loaders import PyPDFLoader 71 | 72 | loader = DirectoryLoader('./docs', glob="**/*.pdf", show_progress=True, loader_cls=PyPDFLoader) 73 | documents = loader.load() 74 | ``` 75 | 76 | 2. Once you've loaded documents, you'll often want to transform them to better suit your application. Langchain comes with a lot of Document Transformers that can help here. 77 | 78 | > Very often we want to split our documents into smaller chunks - pdf files can be large. The recommended class is a **RecursiveCharacterTextSplitter**. It is parameterized by a list of characters. It tries to split on them in order until the chunks are small enough. The default list is **["\n\n", "\n", " ", ""]**. This has the effect of trying to keep all paragraphs (and then sentences, and then words) together as long as possible. 79 | 80 | 3. Let's split our documents into smaller chunks. **chunk_overlap** parameter usually improves RAG capabilities. 81 | 82 | ```python 83 | from langchain_text_splitters import RecursiveCharacterTextSplitter 84 | 85 | text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100) 86 | docs = text_splitter.split_documents(documents) 87 | print(len(docs)) 88 | ``` 89 | 90 | 3. Ok, now let's create qdrant client: 91 | 92 | ```python 93 | from qdrant_client import QdrantClient 94 | 95 | qdrant_client = QdrantClient( 96 | url="", 97 | api_key=userdata.get('qdrant_key') 98 | ) 99 | ``` 100 | 101 | 4. Then, we need to create a collection. Collection is a little bit like a table in a SQL database. We need to specify the size of the vector - 3072. This size is determined by the embedding model we use (**text-embedding-3-large** in our case). 102 | 103 | ```python 104 | from qdrant_client.models import Distance, VectorParams 105 | 106 | qdrant_client.recreate_collection( 107 | collection_name="labor_law", 108 | vectors_config=VectorParams(size=3072, distance=Distance.COSINE), 109 | ) 110 | ``` 111 | 112 | 5. Now, create langchain's vectorstore for Qdrant. Note, that we need to specify the embeddings. 113 | 114 | ```python 115 | from langchain_community.vectorstores import Qdrant 116 | 117 | qdrant = Qdrant(qdrant_client, "labor_law", embeddings) 118 | ``` 119 | 120 | 6. Let's load our documents! 121 | 122 | ```python 123 | qdrant.add_documents(docs) 124 | ``` 125 | 126 | 7. All code in this Task could be also performed in on command: 127 | 128 | ```python 129 | qdrant = Qdrant.from_documents( 130 | docs, 131 | embeddings, 132 | url=, 133 | prefer_grpc=True, 134 | api_key=userdata.get('qdrant_key'), 135 | collection_name="labor_law2", 136 | force_recreate=True 137 | ) 138 | ``` 139 | 140 | 8. Go to Qdrant UI dashboard, select your collection and check if documents are loaded. 141 | 142 | ## Task 4: Query and search vector database 143 | 1. Let's query the database with simple similarity search that return top k most similar documents: 144 | 145 | ```python 146 | query = "Ile przysługuje dni urlopu wypoczynkowego?" 147 | found_docs = qdrant.similarity_search(query, k=3) 148 | print(found_docs[0].page_content) 149 | ``` 150 | >You can also ask the question in english 151 | 152 | 2. You can display the source of each document: 153 | ```python 154 | print(found_docs[0].metadata["source"]) 155 | ``` 156 | 157 | 3. And you can also display a similarity score: 158 | ```python 159 | query = "Ile przysługuje dni urlopu wypoczynkowego?" 160 | found_docs = qdrant.similarity_search_with_score(query, k=3) 161 | for d, s in found_docs: 162 | print(d.page_content) 163 | print(f"\nScore: {s}") 164 | print(d.metadata["source"]) 165 | print("---------------------------------------------------------------") 166 | ``` 167 | 168 | 4. Another useful algorithm for searching is MMR (maximal marginal relevance). 169 | >MMR tries to look for documents most similar to the inputs, while also optimizing for diversity. It does this by finding the examples with the embeddings that have the greatest cosine similarity with the inputs, and then iteratively adding them while penalizing them for closeness to already selected examples. 170 | ```python 171 | query = "Ile przysługuje dni urlopu wypoczynkowego?" 172 | found_docs = qdrant.max_marginal_relevance_search(query, k=3, fetch_k=10) 173 | for d in found_docs: 174 | print(d.page_content) 175 | print(d.metadata["source"]) 176 | print("---------------------------------------------------------------") 177 | 178 | ``` 179 | Compare the results with previous ones. 180 | 181 | 5. Qdrant also allows us to add additional filters. For example we can search only in subset od documents: 182 | 183 | ```python 184 | from qdrant_client import models 185 | 186 | my_filter = models.Filter( 187 | must=[ 188 | models.FieldCondition( 189 | key="metadata.source", 190 | match=models.MatchValue(value="docs/praca_rodzicielstwo.pdf"), 191 | ) 192 | ] 193 | ) 194 | 195 | query = "Ile przysługuje dni urlopu wypoczynkowego?" 196 | 197 | found_docs = qdrant.similarity_search(query, k=3, filter=my_filter) 198 | for d in found_docs: 199 | print(d.page_content) 200 | print(d.metadata["source"]) 201 | print("---------------------------------------------------------------") 202 | 203 | ``` -------------------------------------------------------------------------------- /04_Multi-agent systems/Lab2/README.md: -------------------------------------------------------------------------------- 1 | # Lab 2: Building multi-agent app with Langgraph. 2 | In this lab you will learn how to build a multi-agent app using Langgraph framework. We will build a two agents chat (Researcher and Coder) with common State. There will also be a supervisor which will decide who will talk next: 3 | ![image](./graph.jpeg) 4 | 5 | ## Prerequisutes: 6 | - OpenAI API key with a few Euros credits 7 | - Google account 8 | 9 | ## Task 1: Set-up 10 | 1. Open Google Colab: https://colab.research.google.com/ 11 | 2. Create new notebook, name it eg. **Workshop4 - la2** 12 | 3. First, we need to install dependencies. In the first cell type and run: 13 | 14 | ```python 15 | !pip install --quiet langchain langchain_openai langchain_experimental langsmith langgraph langchainhub==0.1.15 16 | ``` 17 | 18 | 4. Create a configuration for GPT-4o LLM: 19 | 20 | ```python 21 | from google.colab import userdata 22 | from langchain_openai import ChatOpenAI 23 | 24 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 25 | gpt4 = ChatOpenAI(model = "gpt-4o", temperature = 0) 26 | ``` 27 | 28 | 5. And LangSmith configuration: 29 | 30 | ```python 31 | import os 32 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 33 | os.environ["LANGCHAIN_PROJECT"] = "ai-workshops-day4" 34 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 35 | os.environ["LANGCHAIN_API_KEY"] = "" 36 | ``` 37 | 38 | ## Task 2: Create two agents chat. 39 | 40 | 1. First, let's create a researcher agent. He will use Tavily for researching. Here we use **create_react_agent** from Langgraph. That simplifies creating the agents for Langgraph. This method will replace legacy **AgentExecutor** class. 41 | 42 | ```python 43 | from langchain_community.tools.tavily_search import TavilySearchResults 44 | from langchain import hub 45 | from langgraph.prebuilt import create_react_agent 46 | 47 | os.environ["TAVILY_API_KEY"] = userdata.get('tavily_key') 48 | 49 | tavily_tool = TavilySearchResults(max_results=3) 50 | 51 | system_message = "You are a helpful assistant. Your role is to research the web for data and information" 52 | 53 | agent_researcher = create_react_agent(gpt4, [tavily_tool], messages_modifier=system_message) 54 | ``` 55 | 56 | 2. We can test the agent: 57 | 58 | ```python 59 | inputs = {"messages": [("user", "who is the winnner of the man's us open?")]} 60 | for s in agent_researcher.stream(inputs, stream_mode="values"): 61 | message = s["messages"][-1] 62 | if isinstance(message, tuple): 63 | print(message) 64 | else: 65 | message.pretty_print() 66 | ``` 67 | 68 | 3. Now, let's create a coder agent: 69 | 70 | ```python 71 | from langchain_experimental.utilities import PythonREPL 72 | from langchain_core.tools import Tool 73 | 74 | python_repl = PythonREPL() 75 | 76 | repl_tool = Tool( 77 | name="python_repl", 78 | description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.", 79 | func=python_repl.run, 80 | ) 81 | 82 | system_message = "You are a helpful assistant. Your role is to write and execute Python code only." 83 | 84 | agent_coder = create_react_agent(gpt4, [repl_tool], messages_modifier=system_message, debug=True) 85 | ``` 86 | 87 | 4. And test it: 88 | 89 | ```python 90 | inputs = {"messages": [("user", "Write a code that count fibonacci and execute it for n=12")]} 91 | for s in agent_coder.stream(inputs, stream_mode="values"): 92 | message = s["messages"][-1] 93 | if isinstance(message, tuple): 94 | print(message) 95 | else: 96 | message.pretty_print() 97 | ``` 98 | 99 | 5. Now we need supervisor. So a node, that will decide which agent should talk. We will use GPT Tool Calling for this: 100 | 101 | ```python 102 | from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder 103 | from langchain_core.output_parsers.openai_functions import JsonOutputFunctionsParser 104 | 105 | system_prompt = ( 106 | """You are a supervisor tasked with managing a conversation between the 107 | following workers: Researcher, Coder. 108 | Researcher is responsible for gathering data and information from web. 109 | Coder is responsible for code writing and executing. 110 | Given the following user request, respond with the worker to act next. Each worker will perform a 111 | task and respond with their results and status. When a task is done, respond with FINISH.""" 112 | ) 113 | options = ["Researcher", "Coder", "FINISH"] 114 | function_def = { 115 | "name": "route", 116 | "description": "Select the next role.", 117 | "parameters": { 118 | "title": "routeSchema", 119 | "type": "object", 120 | "properties": { 121 | "next": { 122 | "title": "Next", 123 | "anyOf": [ 124 | {"enum": options}, 125 | ], 126 | } 127 | }, 128 | "required": ["next"], 129 | }, 130 | } 131 | prompt = ChatPromptTemplate.from_messages( 132 | [ 133 | ("system", system_prompt), 134 | MessagesPlaceholder(variable_name="messages"), 135 | ( 136 | "system", 137 | "Given the conversation above, who should act next?" 138 | " Or should we FINISH? Select one of: [Researcher, Coder, FINISH]", 139 | ), 140 | ] 141 | ) 142 | 143 | supervisor_chain = ( 144 | prompt 145 | | gpt4.bind_functions(functions=[function_def], function_call="route") 146 | | JsonOutputFunctionsParser() 147 | ) 148 | ``` 149 | 150 | 6. Then, we need a graph state. For now it will contains two variables: messages list and an indicator where to route next: 151 | 152 | ```python 153 | import operator 154 | from typing import Annotated, Any, Dict, List, Optional, Sequence, TypedDict 155 | import functools 156 | from langchain_core.messages import BaseMessage, HumanMessage 157 | from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder 158 | 159 | 160 | class AgentState(TypedDict): 161 | messages: Annotated[Sequence[BaseMessage], operator.add] 162 | # The 'next' field indicates where to route to next 163 | next: str 164 | ``` 165 | 166 | 7. Now we need to create graph nodes. They will execute the agent and update the state. Node for the researcher: 167 | 168 | ```python 169 | def research_node(state: AgentState): 170 | result = agent_researcher.invoke({"messages": state["messages"]}) 171 | return {"messages": [HumanMessage(content=result["messages"][-1].content, name="Researcher")]} 172 | ``` 173 | 174 | 8. Node for coder: 175 | 176 | ```python 177 | def code_node(state: AgentState): 178 | result = agent_coder.invoke({"messages": state["messages"]}) 179 | return {"messages": [HumanMessage(content=result["messages"][-1].content, name="Coder")], } 180 | ``` 181 | 182 | 9. And node for Supervisor: 183 | 184 | ```python 185 | def supervisor_node(state: AgentState): 186 | result = supervisor_chain.invoke({"messages": state["messages"]}) 187 | return {"next": result["next"]} 188 | ``` 189 | 190 | 9. Finally, we can build our graph. Nodes first: 191 | 192 | ```python 193 | from langgraph.graph import StateGraph, END 194 | 195 | workflow = StateGraph(AgentState) 196 | workflow.add_node("Researcher", research_node) 197 | workflow.add_node("Coder", code_node) 198 | workflow.add_node("supervisor", supervisor_node) 199 | ``` 200 | 201 | 10. We need to define the logic about how to conversation will be routed between agents. Here the logic is simple because the **supervisor** indicated who should speak next: 202 | 203 | ```python 204 | def should_continue(state): 205 | if (state["next"] == "FINISH"): 206 | return END 207 | return state["next"] 208 | ``` 209 | 210 | 11. Now, let's add edges. The **add_conditional_edges** should return the name of the next node. 211 | 212 | ```python 213 | workflow.add_edge("Researcher", "supervisor") 214 | workflow.add_edge("Coder", "supervisor") 215 | 216 | workflow.add_conditional_edges("supervisor", should_continue) 217 | ``` 218 | 219 | 12. And entrypoint and compile the graph: 220 | 221 | ```python 222 | workflow.set_entry_point("supervisor") 223 | 224 | graph = workflow.compile() 225 | ``` 226 | 227 | 12. Try to invoke the graph: 228 | 229 | ```python 230 | for s in graph.stream( 231 | { 232 | "messages": [ 233 | HumanMessage(content="Find data about bitcoin price from last 6 months and draw a chart") 234 | ] 235 | } 236 | ): 237 | if "__end__" not in s: 238 | print(s) 239 | print("----") 240 | ``` 241 | 242 | 13. You can also draw the logic: 243 | 244 | ```python 245 | from IPython.display import Image, display 246 | 247 | try: 248 | display(Image(graph.get_graph(xray=True).draw_mermaid_png())) 249 | except: 250 | # This requires some extra dependencies and is optional 251 | pass 252 | ``` -------------------------------------------------------------------------------- /01_Langchain basics/Lab1/README.md: -------------------------------------------------------------------------------- 1 | # Lab 1: Working with Langchain 2 | In this lab you will learn: 3 | - Basics of the Langchain framework 4 | - How to create chains 5 | - How to use LCEL (Langchain expression language) to build different flows of LLM invocations. 6 | - How to use GPT tools (functions) 7 | - How to monitor you app 8 | - How to work with different models (gpt4o, gpt4o-mini, dalle) 9 | 10 | ## Prerequisutes: 11 | - OpenAI API key with a few Euros credits 12 | - Google account 13 | 14 | ## Task 1: First LLM invocations 15 | 1. Open Google Colab: https://colab.research.google.com/ 16 | 1. Create new notebook, name it eg. **Workshop1 - lab1** 17 | 1. In the left menu of Google Colab click **KEY** button. It is a place when you can store your secrets. 18 | 1. Create one new secrets and put your open ai key there: 19 | - openai_key 20 | 1. First, we need to install dependencies. In the first cell type and run:\ 21 | ```!pip install --quiet langchain==0.2.16 langchain-openai==0.1.23 langchain-community==0.2.16```\ 22 | Here we install Langchain framework and langchain-openai responsible for OpenAI integration. 23 | 1. In the next cell paste your OpenAI API key and create an instance of gpt-4o-mini model: 24 | 25 | ```python 26 | import os 27 | from langchain_openai import ChatOpenAI 28 | from google.colab import userdata 29 | 30 | os.environ["OPENAI_API_KEY"] = userdata.get('openai_key') 31 | 32 | llm = ChatOpenAI(model = "gpt-4o-mini") 33 | ``` 34 | **ChatOpenAI** is a Langchain class. You can add here more parameters like temperature, max_tokens etc. [See in docs](https://api.python.langchain.com/en/latest/chat_models/langchain_community.chat_models.openai.ChatOpenAI.html) 35 | 5. Invoke the model with simple prompt: 36 | ```python 37 | llm.invoke("tell ma a joke about pizza") 38 | ``` 39 | 6. Observe the output. Notice tokens usage. 40 | 41 | ## Task 2: Switch on LangSmith. 42 | 43 | LangSmith is an online tool that is used to monitor Langchain applications. It will show you what are exact invocations of the LLM model. Very useful tool. 44 | 45 | 1. Go to: https://smith.langchain.com/ 46 | 2. Sign-up in the service 47 | 3. On the left-hand site click **Settings** 48 | 4. Generate new API key (personal) and copy it 49 | 5. Create new cell in the notebook and paste LangSmith configuration: 50 | ```python 51 | os.environ["LANGCHAIN_TRACING_V2"] = "true" 52 | os.environ["LANGCHAIN_PROJECT"] = "ai-agent-workshops" 53 | os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" 54 | os.environ["LANGCHAIN_API_KEY"] = "YOUR_LANGSMITH_KEY" 55 | ``` 56 | 6. Run one more time the call above that invokes gpt-4o-mini model. 57 | 7. In LangSmith click **Projects** in left menu, find "ai-agent-workshops" project and observe the tracing. 58 | 8. Now LangSmith is switched on. It will be useful in next tasks. Free plan is sufficient. 59 | 60 | ## Task 3: Playing with chains 61 | 1. In a new cell create a prompt template. Prompt templates are useful in order to build a general prompt and modify just a part of it. Here we create a simple **system** and **user** messages with two place holders: **role** and **prompt**: 62 | 63 | ```python 64 | from langchain_core.prompts import ChatPromptTemplate 65 | 66 | prompt = ChatPromptTemplate.from_messages([ 67 | ("system", "You are an assistant that acts as a {role}. Answer only questions about your role"), 68 | ("user", "{prompt}") 69 | ]) 70 | ``` 71 | 2. Most of Langchain classes implements **Runnable** interface so you can run **invoke()** method on it. Let's invoke the prompt and see how placeholders are filled in: 72 | ```python 73 | prompt.invoke({"prompt": "How to make pizza?", "role": "cook"}) 74 | ``` 75 | 2. Notice that our prompt contains two messages: **SystemMessage** and **HumanMessage**. 76 | 2. Now let's create a first chain using LCEL (Langchain expression language). We will use pipe operator (|): 77 | ```python 78 | chain = prompt | llm 79 | chain.invoke({"prompt": "How to make pizza?", "role": "cook"}) 80 | ``` 81 | 2. As a result we get an **AIMessage**. 82 | 2. GPT models are text-2-text model. Very often we want to extract raw text. Langchain comes here with different parsers. Here we will use simple **StrOutpurParser**: 83 | 84 | ```python 85 | from langchain_core.output_parsers import StrOutputParser 86 | 87 | output_parser = StrOutputParser() 88 | chain = prompt | llm | output_parser 89 | chain.invoke({"prompt": "How to make pizza?", "role": "cook"}) 90 | ``` 91 | 7. You can also check other available parsers here: https://python.langchain.com/v0.1/docs/modules/model_io/output_parsers/ 92 | 93 | 8. Try also to invoke the model with non-matching values, for example: `{"prompt": "How to make pizza?", "role": "doctor"}`. :) 94 | 95 | 9. Langchain supports streaming. Just change **invoke()** method to **stream()**: 96 | ```python 97 | for s in chain.stream({"prompt": "How to make pizza?", "role": "cook"}): 98 | print(s, end="", flush=True) 99 | ``` 100 | 10. It is important that each connected **Runnables** (prompt | llm | output_parser) needs to be compatible in terms of schema. You can always check the schema of each chain component: 101 | ```python 102 | print(chain.input_schema.schema()) 103 | print(chain.output_schema.schema()) 104 | ``` 105 | 106 | ## Task 4: Longer chains 107 | 1. Now let's build something a little bit longer and let's switch to GPT-4o. Make an instance of a model: 108 | ```python 109 | gpt4o = ChatOpenAI(model = "gpt-4o") 110 | ``` 111 | 1. Now we will build a chain that will: 112 | - Ask for a dish recipe 113 | - Based on the recipe it will list all needed ingredients 114 | 3. In a new cell, let's create the first chain: 115 | 116 | ```python 117 | recipe_prompt = ChatPromptTemplate.from_messages([ 118 | ("system", "You are a cook that gives dish recipes."), 119 | ("user", "Give me a recipe for {dish}") 120 | ]) 121 | 122 | chain = recipe_prompt | gpt4o | output_parser 123 | print(chain.invoke({"dish": "pizza"})) 124 | ``` 125 | 4. In a new cell, create second prompt: 126 | 127 | ```python 128 | ingredients_prompt = ChatPromptTemplate.from_template("Based on the recipe: {recipe} list me all ingredients needed for the recipe. Return just the list and nothing else") 129 | ``` 130 | 131 | 5. Now create a full chain. Notice that we are using two LLMs here. We are also using simple lambda function to pass output as a new input: 132 | 133 | ```python 134 | chain2 = ( 135 | recipe_prompt 136 | | gpt4o 137 | | output_parser 138 | | (lambda input: {"recipe": input}) 139 | | ingredients_prompt 140 | | llm 141 | | output_parser 142 | ) 143 | print(chain2.invoke({"dish": "pizza"})) 144 | ``` 145 | 6. We can also embed one chain in another: (same effect as above) 146 | 147 | ```python 148 | chain3 = ( 149 | chain 150 | | (lambda input: {"recipe": input}) 151 | | ingredients_prompt 152 | | llm 153 | | output_parser 154 | ) 155 | print(chain3.invoke({"dish": "pizza"})) 156 | ``` 157 | 7. Investigate LangSmith debug 158 | 159 | ## Task 5: Tool calling 160 | Most of new models support tools calling (previously called function calling). This mechanism is used to return structured output from LLM. Remember that this mechanism does not call any function/tool. It just prepares the input. 161 | 162 | 1. In Langchain we can define tools in several ways. For example using @tool decorator or using Pydantic types. Here we define two tools (**add** and **multiply**) in two different ways: 163 | 164 | ```python 165 | from langchain_core.tools import tool 166 | from langchain_core.pydantic_v1 import BaseModel, Field 167 | 168 | # add tool 169 | @tool 170 | def add(a: int, b: int) -> int: 171 | "Add two numbers" 172 | return a + b 173 | 174 | # multiply tool 175 | class Multiply(BaseModel): 176 | a: int = Field(..., description="First integer") 177 | b: int = Field(..., description="Second integer") 178 | 179 | def execute(self) -> int: 180 | return self.a * self.b 181 | 182 | @tool 183 | def multiply_tool(data: Multiply) -> int: 184 | "Multiply two numbers" 185 | return data.execute() 186 | 187 | tools = [add, multiply_tool] 188 | ``` 189 | 2. Now we need to bind these tools to our LLM so it can invoke them. Note, that by default LLM will decide if it wants to use a tool or not. We can also enforce it. 190 | 191 | ```python 192 | llm_with_tools = llm.bind_tools(tools) 193 | ``` 194 | 195 | 3. Let's call it and observe how LLM model has used the tools: 196 | ```python 197 | result = llm_with_tools.invoke("What is 3 * 8? Also, what is 8 + 63?") 198 | result 199 | ``` 200 | 201 | 4. In a new cell inspect tool calls: 202 | ```python 203 | result.tool_calls 204 | ``` 205 | 206 | 5. We can also explicitly invoke a tool function with arguments. Let's extract them from the result and invoke the **add** tool: 207 | 208 | ```python 209 | def extract_args(functions, name): 210 | for func in functions: 211 | if func['name'] == name: 212 | return func['args'] 213 | return None 214 | 215 | add_args = extract_args(result.tool_calls, 'add') 216 | add.invoke(add_args) 217 | ``` 218 | 219 | 6. Tool calling are also traced by LangSmith. 220 | 221 | ## Task 6: Image generation with Dall-e 222 | 223 | 224 | In Lanchain we have a **DallEAPIWrapper** class for using Dall-e model. In the below example we will first use gpt-4o-mini to generate a prompt for Dall-e and then generate an image. Note that Dall-e 3 accepts prompt of maximum 4000 characters. That is why we will limit max_token to 800. 225 | 226 | 1. In a new cell type: 227 | 228 | ```python 229 | from langchain_community.utilities.dalle_image_generator import DallEAPIWrapper 230 | 231 | llm = ChatOpenAI(max_tokens=800) 232 | 233 | prompt = ChatPromptTemplate.from_template("Generate a detailed prompt to generate an image based on the following description: {image_desc}.") 234 | dalle_chain = prompt | llm | output_parser 235 | dalle_prompt = dalle_chain.invoke({"image_desc": "cat at home"}) 236 | print(dalle_prompt) 237 | ``` 238 | 239 | 2. Now let's define a function that displays image in a notebook: 240 | 241 | ```python 242 | def display_image(image_url): 243 | try: 244 | import google.colab 245 | IN_COLAB = True 246 | except ImportError: 247 | IN_COLAB = False 248 | 249 | if IN_COLAB: 250 | from google.colab.patches import cv2_imshow # for image display 251 | from skimage import io 252 | import cv2 253 | 254 | image = io.imread(image_url) 255 | image_bgr = cv2.cvtColor(image, cv2.COLOR_RGB2BGR) # Convert RGB to BGR 256 | cv2_imshow(image_bgr) 257 | else: 258 | import cv2 259 | from skimage import io 260 | 261 | image = io.imread(image_url) 262 | image_bgr = cv2.cvtColor(image, cv2.COLOR_RGB2BGR) # Convert RGB to BGR 263 | cv2.imshow("image", image_bgr) 264 | cv2.waitKey(0) # wait for a keyboard input 265 | cv2.destroyAllWindows() 266 | ``` 267 | 268 | 3. And finally let's call dall-e and display the image: 269 | 270 | ```python 271 | image_url = DallEAPIWrapper(model = "dall-e-3").run(dalle_prompt) 272 | print(image_url) 273 | display_image(image_url) 274 | ``` 275 | image_url variable will point to a public image on Azure blob storage. 276 | 277 | 4. (For fun) You can change the dall-e model to 'dall-e-2' and observe the difference :) 278 | 279 | ## End lab 280 | --------------------------------------------------------------------------------