├── LLAMA3_AGent.png ├── agent-overview.png ├── langsmith_rag_eval.png ├── plan-and-execute.png ├── README.md ├── Autogen_Research_Agent.ipynb ├── Autogen_planning_with_Agents.ipynb └── Autogen_groupchat_RAG.ipynb /LLAMA3_AGent.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NisaarAgharia/AI-Agents/HEAD/LLAMA3_AGent.png -------------------------------------------------------------------------------- /agent-overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NisaarAgharia/AI-Agents/HEAD/agent-overview.png -------------------------------------------------------------------------------- /langsmith_rag_eval.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NisaarAgharia/AI-Agents/HEAD/langsmith_rag_eval.png -------------------------------------------------------------------------------- /plan-and-execute.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NisaarAgharia/AI-Agents/HEAD/plan-and-execute.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Design Patterns for Autonomous AI Agents and Various Frameworks Like Autogen, Langraph, Taskweaver 2 | 3 | ## Agent Architecture 4 | ![agent-overview](https://github.com/NisaarAgharia/AI-Agents/assets/22457544/af916581-eb72-4412-a208-77de24ed7d23) 5 | 6 | ## Planner Component: 7 | ![plan-and-execute](https://github.com/NisaarAgharia/Design-AI-Agents/assets/22457544/4361972e-acaa-427a-96c7-37d842c42448) 8 | 9 | ## Taskweaver Design: 10 | taskweaver_arch 11 | [GitHub Repository](https://github.com/microsoft/taskweaver/) 12 | 13 | ## Autogen Agent Chat 14 | ![autogen_agentchat](https://github.com/NisaarAgharia/AI-Agents/assets/22457544/1cd7ba29-2ad5-4fa6-94b2-c5e45a6c7eb9) 15 | [GitHub Repository](https://github.com/microsoft/autogen) 16 | 17 | ## Multi-Agent Conversation 18 | chat_example 19 | 20 | ## Hugging GPT Agent 21 | hugging-gpt 22 | -------------------------------------------------------------------------------- /Autogen_Research_Agent.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "attachments": {}, 5 | "cell_type": "markdown", 6 | "metadata": {}, 7 | "source": [ 8 | "# Perform Research with Multi-Agent Group Chat\n", 9 | "\n", 10 | "AutoGen offers conversable agents powered by LLM, tool, or human, which can be used to perform tasks collectively via automated chat. This framework allows tool use and human participation through multi-agent conversation.\n", 11 | "Please find documentation about this feature [here](https://microsoft.github.io/autogen/docs/Use-Cases/agent_chat).\n", 12 | "\n", 13 | "## Requirements\n", 14 | "\n", 15 | "````{=mdx}\n", 16 | ":::info Requirements\n", 17 | "Install `pyautogen`:\n", 18 | "```bash\n", 19 | "pip install pyautogen\n", 20 | "```\n", 21 | "\n", 22 | "For more information, please refer to the [installation guide](/docs/installation/).\n", 23 | ":::\n", 24 | "````" 25 | ] 26 | }, 27 | { 28 | "attachments": {}, 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "## Set your API Endpoint\n", 33 | "\n", 34 | "The [`config_list_from_json`](https://microsoft.github.io/autogen/docs/reference/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file." 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": 2, 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "import autogen\n", 44 | "\n", 45 | "config_list_gpt4 = autogen.config_list_from_json(\n", 46 | " \"OAI_CONFIG_LIST\",\n", 47 | " filter_dict={\n", 48 | " \"model\": [\"gpt-4-32k\", \"gpt-4-32k-0314\", \"gpt-4-32k-v0314\"],\n", 49 | " },\n", 50 | ")" 51 | ] 52 | }, 53 | { 54 | "attachments": {}, 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "````{=mdx}\n", 59 | ":::tip\n", 60 | "Learn more about configuring LLMs for agents [here](/docs/topics/llm_configuration).\n", 61 | ":::\n", 62 | "````" 63 | ] 64 | }, 65 | { 66 | "attachments": {}, 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "## Construct Agents" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": 3, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "gpt4_config = {\n", 80 | " \"cache_seed\": 42, # change the cache_seed for different trials\n", 81 | " \"temperature\": 0,\n", 82 | " \"config_list\": config_list_gpt4,\n", 83 | " \"timeout\": 120,\n", 84 | "}\n", 85 | "user_proxy = autogen.UserProxyAgent(\n", 86 | " name=\"Admin\",\n", 87 | " system_message=\"A human admin. Interact with the planner to discuss the plan. Plan execution needs to be approved by this admin.\",\n", 88 | " code_execution_config=False,\n", 89 | ")\n", 90 | "engineer = autogen.AssistantAgent(\n", 91 | " name=\"Engineer\",\n", 92 | " llm_config=gpt4_config,\n", 93 | " system_message=\"\"\"Engineer. You follow an approved plan. You write python/shell code to solve tasks. Wrap the code in a code block that specifies the script type. The user can't modify your code. So do not suggest incomplete code which requires others to modify. Don't use a code block if it's not intended to be executed by the executor.\n", 94 | "Don't include multiple code blocks in one response. Do not ask others to copy and paste the result. Check the execution result returned by the executor.\n", 95 | "If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try.\n", 96 | "\"\"\",\n", 97 | ")\n", 98 | "scientist = autogen.AssistantAgent(\n", 99 | " name=\"Scientist\",\n", 100 | " llm_config=gpt4_config,\n", 101 | " system_message=\"\"\"Scientist. You follow an approved plan. You are able to categorize papers after seeing their abstracts printed. You don't write code.\"\"\",\n", 102 | ")\n", 103 | "planner = autogen.AssistantAgent(\n", 104 | " name=\"Planner\",\n", 105 | " system_message=\"\"\"Planner. Suggest a plan. Revise the plan based on feedback from admin and critic, until admin approval.\n", 106 | "The plan may involve an engineer who can write code and a scientist who doesn't write code.\n", 107 | "Explain the plan first. Be clear which step is performed by an engineer, and which step is performed by a scientist.\n", 108 | "\"\"\",\n", 109 | " llm_config=gpt4_config,\n", 110 | ")\n", 111 | "executor = autogen.UserProxyAgent(\n", 112 | " name=\"Executor\",\n", 113 | " system_message=\"Executor. Execute the code written by the engineer and report the result.\",\n", 114 | " human_input_mode=\"NEVER\",\n", 115 | " code_execution_config={\n", 116 | " \"last_n_messages\": 3,\n", 117 | " \"work_dir\": \"paper\",\n", 118 | " \"use_docker\": False,\n", 119 | " }, # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n", 120 | ")\n", 121 | "critic = autogen.AssistantAgent(\n", 122 | " name=\"Critic\",\n", 123 | " system_message=\"Critic. Double check plan, claims, code from other agents and provide feedback. Check whether the plan includes adding verifiable info such as source URL.\",\n", 124 | " llm_config=gpt4_config,\n", 125 | ")\n", 126 | "groupchat = autogen.GroupChat(\n", 127 | " agents=[user_proxy, engineer, scientist, planner, executor, critic], messages=[], max_round=50\n", 128 | ")\n", 129 | "manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=gpt4_config)" 130 | ] 131 | }, 132 | { 133 | "attachments": {}, 134 | "cell_type": "markdown", 135 | "metadata": {}, 136 | "source": [ 137 | "## Start Chat" 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": 4, 143 | "metadata": {}, 144 | "outputs": [ 145 | { 146 | "name": "stdout", 147 | "output_type": "stream", 148 | "text": [ 149 | "\u001b[33mAdmin\u001b[0m (to chat_manager):\n", 150 | "\n", 151 | "\n", 152 | "find papers on LLM applications from arxiv in the last week, create a markdown table of different domains.\n", 153 | "\n", 154 | "\n", 155 | "--------------------------------------------------------------------------------\n", 156 | "\u001b[33mPlanner\u001b[0m (to chat_manager):\n", 157 | "\n", 158 | "Plan:\n", 159 | "\n", 160 | "1. Engineer: Write a script to scrape the arXiv website for papers related to LLM (Language Model) applications published in the last week. The script should extract the title, authors, abstract, and link to the paper.\n", 161 | "\n", 162 | "2. Scientist: Review the scraped data to identify the different domains in which LLM is applied. This could be based on keywords in the title or abstract, or the scientist's knowledge of the field.\n", 163 | "\n", 164 | "3. Engineer: Modify the script to categorize the papers based on the domains identified by the scientist. The script should output a markdown table with columns for the domain, title, authors, abstract, and link.\n", 165 | "\n", 166 | "4. Scientist: Review the markdown table to ensure the papers are correctly categorized and the information is accurate.\n", 167 | "\n", 168 | "5. Engineer: Make any necessary revisions to the script based on the scientist's feedback.\n", 169 | "\n", 170 | "6. Scientist: Give final approval of the markdown table.\n", 171 | "\n", 172 | "7. Engineer: Submit the final markdown table.\n", 173 | "\n", 174 | "--------------------------------------------------------------------------------\n", 175 | "\u001b[33mCritic\u001b[0m (to chat_manager):\n", 176 | "\n", 177 | "The plan seems solid and well-structured. However, it lacks the inclusion of verifiable information such as source URLs. Here's a revised version:\n", 178 | "\n", 179 | "1. Engineer: Write a script to scrape the arXiv website for papers related to LLM (Language Model) applications published in the last week. The script should extract the title, authors, abstract, and link to the paper.\n", 180 | "\n", 181 | "2. Scientist: Review the scraped data to identify the different domains in which LLM is applied. This could be based on keywords in the title or abstract, or the scientist's knowledge of the field.\n", 182 | "\n", 183 | "3. Engineer: Modify the script to categorize the papers based on the domains identified by the scientist. The script should output a markdown table with columns for the domain, title, authors, abstract, and link (source URL).\n", 184 | "\n", 185 | "4. Scientist: Review the markdown table to ensure the papers are correctly categorized and the information is accurate. Check the source URLs to verify the information.\n", 186 | "\n", 187 | "5. Engineer: Make any necessary revisions to the script based on the scientist's feedback.\n", 188 | "\n", 189 | "6. Scientist: Give final approval of the markdown table, ensuring all entries have a valid source URL.\n", 190 | "\n", 191 | "7. Engineer: Submit the final markdown table.\n", 192 | "\n", 193 | "--------------------------------------------------------------------------------\n", 194 | "\u001b[31m\n", 195 | ">>>>>>>> NO HUMAN INPUT RECEIVED.\u001b[0m\n", 196 | "\u001b[31m\n", 197 | ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", 198 | "\u001b[33mAdmin\u001b[0m (to chat_manager):\n", 199 | "\n", 200 | "\n", 201 | "\n", 202 | "--------------------------------------------------------------------------------\n", 203 | "\u001b[33mEngineer\u001b[0m (to chat_manager):\n", 204 | "\n", 205 | "I'm sorry for the confusion, but as an AI model developed by OpenAI, I don't have the ability to directly scrape websites or access real-time data from external databases or websites, including arXiv. However, I can help you write a Python script that uses the arXiv API to fetch the data you need.\n", 206 | "\n", 207 | "Here's a Python script that uses the `arxiv` package to search for papers related to \"LLM applications\" from the last week. This script will print out the title, authors, summary, and URL of each paper.\n", 208 | "\n", 209 | "```python\n", 210 | "import arxiv\n", 211 | "import datetime\n", 212 | "\n", 213 | "# Calculate the date one week ago\n", 214 | "one_week_ago = (datetime.datetime.now() - datetime.timedelta(days=7)).strftime('%Y%m%d%H%M%S')\n", 215 | "\n", 216 | "# Search for papers on LLM applications\n", 217 | "search = arxiv.Search(\n", 218 | " query=\"LLM applications\",\n", 219 | " max_results=100,\n", 220 | " sort_by=arxiv.SortCriterion.SubmittedDate,\n", 221 | " sort_order=arxiv.SortOrder.Descending\n", 222 | ")\n", 223 | "\n", 224 | "for result in search.get():\n", 225 | " # Check if the paper was submitted in the last week\n", 226 | " if result.submitted.strftime('%Y%m%d%H%M%S') > one_week_ago:\n", 227 | " print(f\"Title: {result.title}\")\n", 228 | " print(f\"Authors: {', '.join(author.name for author in result.authors)}\")\n", 229 | " print(f\"Summary: {result.summary}\")\n", 230 | " print(f\"URL: {result.entry_id}\")\n", 231 | " print(\"\\n\")\n", 232 | "```\n", 233 | "\n", 234 | "You can install the `arxiv` package using pip:\n", 235 | "\n", 236 | "```shell\n", 237 | "pip install arxiv\n", 238 | "```\n", 239 | "\n", 240 | "Please note that you'll need to replace `\"LLM applications\"` with the actual search term you're interested in. The arXiv API doesn't support searching by the submission date, so the script fetches the 100 most recent results and filters them manually. You may need to increase `max_results` if there are more than 100 papers submitted per week on your topic.\n", 241 | "\n", 242 | "To create a markdown table, you'll need to format the output like this:\n", 243 | "\n", 244 | "```markdown\n", 245 | "| Title | Authors | Summary | URL |\n", 246 | "|-------|---------|---------|-----|\n", 247 | "| ... | ... | ... | ... |\n", 248 | "```\n", 249 | "\n", 250 | "You can do this in Python using f-strings:\n", 251 | "\n", 252 | "```python\n", 253 | "print(\"| Title | Authors | Summary | URL |\")\n", 254 | "print(\"|-------|---------|---------|-----|\")\n", 255 | "for result in search.get():\n", 256 | " # Check if the paper was submitted in the last week\n", 257 | " if result.submitted.strftime('%Y%m%d%H%M%S') > one_week_ago:\n", 258 | " print(f\"| {result.title} | {', '.join(author.name for author in result.authors)} | {result.summary} | {result.entry_id} |\")\n", 259 | "```\n", 260 | "\n", 261 | "This will print a markdown table to the console. If you want to write the table to a file, you can open a file and use the `write` method instead of `print`.\n", 262 | "\n", 263 | "--------------------------------------------------------------------------------\n", 264 | "\u001b[31m\n", 265 | ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n", 266 | "\u001b[33mExecutor\u001b[0m (to chat_manager):\n", 267 | "\n", 268 | "exitcode: 1 (execution failed)\n", 269 | "Code output: \n", 270 | ":15: DeprecationWarning: The 'get' method is deprecated, use 'results' instead\n", 271 | " for result in search.get():\n", 272 | "Traceback (most recent call last):\n", 273 | " File \"\", line 17, in \n", 274 | " if result.submitted.strftime('%Y%m%d%H%M%S') > one_week_ago:\n", 275 | "AttributeError: 'Result' object has no attribute 'submitted'\n", 276 | "\n", 277 | "\n", 278 | "--------------------------------------------------------------------------------\n", 279 | "\u001b[33mEngineer\u001b[0m (to chat_manager):\n", 280 | "\n", 281 | "I apologize for the confusion. It seems there was a mistake in the code. The `arxiv` package's `Result` object doesn't have a `submitted` attribute. Instead, we should use the `published` attribute to get the date when the paper was published.\n", 282 | "\n", 283 | "Here's the corrected code:\n", 284 | "\n", 285 | "```python\n", 286 | "import arxiv\n", 287 | "import datetime\n", 288 | "\n", 289 | "# Calculate the date one week ago\n", 290 | "one_week_ago = (datetime.datetime.now() - datetime.timedelta(days=7)).strftime('%Y%m%d%H%M%S')\n", 291 | "\n", 292 | "# Search for papers on LLM applications\n", 293 | "search = arxiv.Search(\n", 294 | " query=\"LLM applications\",\n", 295 | " max_results=100,\n", 296 | " sort_by=arxiv.SortCriterion.Relevance,\n", 297 | " sort_order=arxiv.SortOrder.Descending\n", 298 | ")\n", 299 | "\n", 300 | "print(\"| Title | Authors | Summary | URL |\")\n", 301 | "print(\"|-------|---------|---------|-----|\")\n", 302 | "for result in search.results():\n", 303 | " # Check if the paper was published in the last week\n", 304 | " if result.published.strftime('%Y%m%d%H%M%S') > one_week_ago:\n", 305 | " print(f\"| {result.title} | {', '.join(author.name for author in result.authors)} | {result.summary} | {result.entry_id} |\")\n", 306 | "```\n", 307 | "\n", 308 | "This script will print a markdown table of papers related to \"LLM applications\" that were published in the last week. The table includes the title, authors, summary, and URL of each paper.\n", 309 | "\n", 310 | "--------------------------------------------------------------------------------\n", 311 | "\u001b[31m\n", 312 | ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n", 313 | "\u001b[33mExecutor\u001b[0m (to chat_manager):\n", 314 | "\n", 315 | "exitcode: 0 (execution succeeded)\n", 316 | "Code output: \n", 317 | "| Title | Authors | Summary | URL |\n", 318 | "|-------|---------|---------|-----|\n", 319 | "| Large Language Models as Data Preprocessors | Haochen Zhang, Yuyang Dong, Chuan Xiao, Masafumi Oyamada | Large Language Models (LLMs), typified by OpenAI's GPT series and Meta's\n", 320 | "LLaMA variants, have marked a significant advancement in artificial\n", 321 | "intelligence. Trained on vast amounts of text data, LLMs are capable of\n", 322 | "understanding and generating human-like text across a diverse range of topics.\n", 323 | "This study expands on the applications of LLMs, exploring their potential in\n", 324 | "data preprocessing, a critical stage in data mining and analytics applications.\n", 325 | "We delve into the applicability of state-of-the-art LLMs such as GPT-3.5,\n", 326 | "GPT-4, and Vicuna-13B for error detection, data imputation, schema matching,\n", 327 | "and entity matching tasks. Alongside showcasing the inherent capabilities of\n", 328 | "LLMs, we highlight their limitations, particularly in terms of computational\n", 329 | "expense and inefficiency. We propose an LLM-based framework for data\n", 330 | "preprocessing, which integrates cutting-edge prompt engineering techniques,\n", 331 | "coupled with traditional methods like contextualization and feature selection,\n", 332 | "to improve the performance and efficiency of these models. The effectiveness of\n", 333 | "LLMs in data preprocessing is evaluated through an experimental study spanning\n", 334 | "12 datasets. GPT-4 emerged as a standout, achieving 100\\% accuracy or F1 score\n", 335 | "on 4 datasets, suggesting LLMs' immense potential in these tasks. Despite\n", 336 | "certain limitations, our study underscores the promise of LLMs in this domain\n", 337 | "and anticipates future developments to overcome current hurdles. | http://arxiv.org/abs/2308.16361v1 |\n", 338 | "| Large language models in medicine: the potentials and pitfalls | Jesutofunmi A. Omiye, Haiwen Gui, Shawheen J. Rezaei, James Zou, Roxana Daneshjou | Large language models (LLMs) have been applied to tasks in healthcare,\n", 339 | "ranging from medical exam questions to responding to patient questions. With\n", 340 | "increasing institutional partnerships between companies producing LLMs and\n", 341 | "healthcare systems, real world clinical application is coming closer to\n", 342 | "reality. As these models gain traction, it is essential for healthcare\n", 343 | "practitioners to understand what LLMs are, their development, their current and\n", 344 | "potential applications, and the associated pitfalls when utilized in medicine.\n", 345 | "This review and accompanying tutorial aim to give an overview of these topics\n", 346 | "to aid healthcare practitioners in understanding the rapidly changing landscape\n", 347 | "of LLMs as applied to medicine. | http://arxiv.org/abs/2309.00087v1 |\n", 348 | "| Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following | Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li, Pheng-Ann Heng | We introduce Point-Bind, a 3D multi-modality model aligning point clouds with\n", 349 | "2D image, language, audio, and video. Guided by ImageBind, we construct a joint\n", 350 | "embedding space between 3D and multi-modalities, enabling many promising\n", 351 | "applications, e.g., any-to-3D generation, 3D embedding arithmetic, and 3D\n", 352 | "open-world understanding. On top of this, we further present Point-LLM, the\n", 353 | "first 3D large language model (LLM) following 3D multi-modal instructions. By\n", 354 | "parameter-efficient fine-tuning techniques, Point-LLM injects the semantics of\n", 355 | "Point-Bind into pre-trained LLMs, e.g., LLaMA, which requires no 3D instruction\n", 356 | "data, but exhibits superior 3D and multi-modal question-answering capacity. We\n", 357 | "hope our work may cast a light on the community for extending 3D point clouds\n", 358 | "to multi-modality applications. Code is available at\n", 359 | "https://github.com/ZiyuGuo99/Point-Bind_Point-LLM. | http://arxiv.org/abs/2309.00615v1 |\n", 360 | "| Where Would I Go Next? Large Language Models as Human Mobility Predictors | Xinglei Wang, Meng Fang, Zichao Zeng, Tao Cheng | Accurate human mobility prediction underpins many important applications\n", 361 | "across a variety of domains, including epidemic modelling, transport planning,\n", 362 | "and emergency responses. Due to the sparsity of mobility data and the\n", 363 | "stochastic nature of people's daily activities, achieving precise predictions\n", 364 | "of people's locations remains a challenge. While recently developed large\n", 365 | "language models (LLMs) have demonstrated superior performance across numerous\n", 366 | "language-related tasks, their applicability to human mobility studies remains\n", 367 | "unexplored. Addressing this gap, this article delves into the potential of LLMs\n", 368 | "for human mobility prediction tasks. We introduce a novel method, LLM-Mob,\n", 369 | "which leverages the language understanding and reasoning capabilities of LLMs\n", 370 | "for analysing human mobility data. We present concepts of historical stays and\n", 371 | "context stays to capture both long-term and short-term dependencies in human\n", 372 | "movement and enable time-aware prediction by using time information of the\n", 373 | "prediction target. Additionally, we design context-inclusive prompts that\n", 374 | "enable LLMs to generate more accurate predictions. Comprehensive evaluations of\n", 375 | "our method reveal that LLM-Mob excels in providing accurate and interpretable\n", 376 | "predictions, highlighting the untapped potential of LLMs in advancing human\n", 377 | "mobility prediction techniques. We posit that our research marks a significant\n", 378 | "paradigm shift in human mobility modelling, transitioning from building complex\n", 379 | "domain-specific models to harnessing general-purpose LLMs that yield accurate\n", 380 | "predictions through language instructions. The code for this work is available\n", 381 | "at https://github.com/xlwang233/LLM-Mob. | http://arxiv.org/abs/2308.15197v1 |\n", 382 | "| Interactively Robot Action Planning with Uncertainty Analysis and Active Questioning by Large Language Model | Kazuki Hori, Kanata Suzuki, Tetsuya Ogata | The application of the Large Language Model (LLM) to robot action planning\n", 383 | "has been actively studied. The instructions given to the LLM by natural\n", 384 | "language may include ambiguity and lack of information depending on the task\n", 385 | "context. It is possible to adjust the output of LLM by making the instruction\n", 386 | "input more detailed; however, the design cost is high. In this paper, we\n", 387 | "propose the interactive robot action planning method that allows the LLM to\n", 388 | "analyze and gather missing information by asking questions to humans. The\n", 389 | "method can minimize the design cost of generating precise robot instructions.\n", 390 | "We demonstrated the effectiveness of our method through concrete examples in\n", 391 | "cooking tasks. However, our experiments also revealed challenges in robot\n", 392 | "action planning with LLM, such as asking unimportant questions and assuming\n", 393 | "crucial information without asking. Shedding light on these issues provides\n", 394 | "valuable insights for future research on utilizing LLM for robotics. | http://arxiv.org/abs/2308.15684v1 |\n", 395 | "| AskIt: Unified Programming Interface for Programming with Large Language Models | Katsumi Okuda, Saman Amarasinghe | In the evolving landscape of software development, Large Language Models\n", 396 | "(LLMs) exhibit a unique phenomenon known as emergent abilities, demonstrating\n", 397 | "adeptness across numerous tasks, from text summarization to code generation.\n", 398 | "While these abilities open up novel avenues in software design and crafting,\n", 399 | "their incorporation presents substantial challenges. Developers grapple with\n", 400 | "decisions surrounding the direct embedding of LLMs within applications versus\n", 401 | "employing them for code generation. Moreover, effective prompt design becomes a\n", 402 | "critical concern, given the necessity of data extraction from natural language\n", 403 | "outputs. To address these intricacies, this paper introduces AskIt, a\n", 404 | "domain-specific language (DSL) specifically designed for LLMs. AskIt simplifies\n", 405 | "LLM integration, offering type-guided output control, template-based function\n", 406 | "definitions, and a unified interface that diminishes the distinction between\n", 407 | "LLM-based code generation and application integration. Furthermore, through\n", 408 | "Programming by Example (PBE), AskIt harnesses the power of few-shot learning at\n", 409 | "the programming language level. Our evaluations underscore AskIt's potency.\n", 410 | "Across 50 tasks, AskIt generated concise prompts for the given tasks, achieving\n", 411 | "a 16.14% reduction in prompt length relative to benchmarks. Additionally, by\n", 412 | "enabling the transition from direct LLM application usage to function\n", 413 | "generation, AskIt achieved significant speedups, as observed in our GSM8K\n", 414 | "benchmark experiments. Through these advancements, AskIt streamlines the\n", 415 | "integration of LLMs in software development, offering a more efficient,\n", 416 | "versatile approach for leveraging emergent abilities. The implementations of\n", 417 | "AskIt in TypeScript and Python are available at\n", 418 | "https://github.com/katsumiok/ts-askit and https://github.com/katsumiok/pyaskit,\n", 419 | "respectively. | http://arxiv.org/abs/2308.15645v1 |\n", 420 | "\n", 421 | "\n", 422 | "--------------------------------------------------------------------------------\n", 423 | "\u001b[33mScientist\u001b[0m (to chat_manager):\n", 424 | "\n", 425 | "Here are the papers related to LLM applications published in the last week:\n", 426 | "\n", 427 | "| Domain | Title | Authors | Summary | URL |\n", 428 | "|-------|---------|---------|-----|-----|\n", 429 | "| Data Preprocessing | [Large Language Models as Data Preprocessors](http://arxiv.org/abs/2308.16361v1) | Haochen Zhang, Yuyang Dong, Chuan Xiao, Masafumi Oyamada | This study expands on the applications of LLMs, exploring their potential in data preprocessing, a critical stage in data mining and analytics applications. | [Link](http://arxiv.org/abs/2308.16361v1) |\n", 430 | "| Medicine | [Large language models in medicine: the potentials and pitfalls](http://arxiv.org/abs/2309.00087v1) | Jesutofunmi A. Omiye, Haiwen Gui, Shawheen J. Rezaei, James Zou, Roxana Daneshjou | This review and accompanying tutorial aim to give an overview of these topics to aid healthcare practitioners in understanding the rapidly changing landscape of LLMs as applied to medicine. | [Link](http://arxiv.org/abs/2309.00087v1) |\n", 431 | "| 3D Understanding, Generation, and Instruction Following | [Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following](http://arxiv.org/abs/2309.00615v1) | Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li, Pheng-Ann Heng | We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video. | [Link](http://arxiv.org/abs/2309.00615v1) |\n", 432 | "| Human Mobility Prediction | [Where Would I Go Next? Large Language Models as Human Mobility Predictors](http://arxiv.org/abs/2308.15197v1) | Xinglei Wang, Meng Fang, Zichao Zeng, Tao Cheng | This article delves into the potential of LLMs for human mobility prediction tasks. | [Link](http://arxiv.org/abs/2308.15197v1) |\n", 433 | "| Robotics | [Interactively Robot Action Planning with Uncertainty Analysis and Active Questioning by Large Language Model](http://arxiv.org/abs/2308.15684v1) | Kazuki Hori, Kanata Suzuki, Tetsuya Ogata | In this paper, we propose the interactive robot action planning method that allows the LLM to analyze and gather missing information by asking questions to humans. | [Link](http://arxiv.org/abs/2308.15684v1) |\n", 434 | "| Software Development | [AskIt: Unified Programming Interface for Programming with Large Language Models](http://arxiv.org/abs/2308.15645v1) | Katsumi Okuda, Saman Amarasinghe | This paper introduces AskIt, a domain-specific language (DSL) specifically designed for LLMs. | [Link](http://arxiv.org/abs/2308.15645v1) |\n", 435 | "\n", 436 | "--------------------------------------------------------------------------------\n", 437 | "\u001b[33mCritic\u001b[0m (to chat_manager):\n", 438 | "\n", 439 | "The scientist has done a good job categorizing the papers into different domains and providing a summary for each. The markdown table is correctly formatted and includes the source URL for each paper, which allows for verification of the information. The domains identified are diverse, indicating a broad range of applications for Large Language Models (LLMs). This review and categorization should provide a useful resource for anyone interested in the recent applications of LLMs.\n", 440 | "\n", 441 | "--------------------------------------------------------------------------------\n" 442 | ] 443 | } 444 | ], 445 | "source": [ 446 | "user_proxy.initiate_chat(\n", 447 | " manager,\n", 448 | " message=\"\"\"\n", 449 | "find papers on LLM applications from arxiv in the last week, create a markdown table of different domains.\n", 450 | "\"\"\",\n", 451 | ")" 452 | ] 453 | }, 454 | { 455 | "attachments": {}, 456 | "cell_type": "markdown", 457 | "metadata": {}, 458 | "source": [ 459 | "## Create Group Chat without Critic for Comparison" 460 | ] 461 | }, 462 | { 463 | "cell_type": "code", 464 | "execution_count": 5, 465 | "metadata": {}, 466 | "outputs": [ 467 | { 468 | "name": "stdout", 469 | "output_type": "stream", 470 | "text": [ 471 | "\u001b[33mAdmin\u001b[0m (to chat_manager):\n", 472 | "\n", 473 | "\n", 474 | "find papers on LLM applications from arxiv in the last week, create a markdown table of different domains.\n", 475 | "\n", 476 | "\n", 477 | "--------------------------------------------------------------------------------\n", 478 | "\u001b[33mPlanner\u001b[0m (to chat_manager):\n", 479 | "\n", 480 | "Plan:\n", 481 | "\n", 482 | "1. Engineer: Write a script to scrape the arXiv website for papers related to LLM (Language Model) applications published in the last week. The script should extract the title, authors, abstract, and link to the paper.\n", 483 | "\n", 484 | "2. Scientist: Review the scraped data to identify the different domains in which LLM is applied. This could be based on keywords in the title or abstract, or the scientist's knowledge of the field.\n", 485 | "\n", 486 | "3. Engineer: Modify the script to categorize the papers based on the domains identified by the scientist. The script should output a markdown table with columns for the domain, title, authors, abstract, and link.\n", 487 | "\n", 488 | "4. Scientist: Review the markdown table to ensure the papers are correctly categorized and the information is accurate.\n", 489 | "\n", 490 | "5. Engineer: Make any necessary revisions to the script based on the scientist's feedback.\n", 491 | "\n", 492 | "6. Scientist: Give final approval of the markdown table.\n", 493 | "\n", 494 | "7. Engineer: Submit the final markdown table.\n", 495 | "\n", 496 | "--------------------------------------------------------------------------------\n" 497 | ] 498 | } 499 | ], 500 | "source": [ 501 | "groupchat_nocritic = autogen.GroupChat(\n", 502 | " agents=[user_proxy, engineer, scientist, planner, executor], messages=[], max_round=50\n", 503 | ")\n", 504 | "for agent in groupchat.agents:\n", 505 | " agent.reset()\n", 506 | "manager_nocritic = autogen.GroupChatManager(groupchat=groupchat_nocritic, llm_config=gpt4_config)\n", 507 | "user_proxy.initiate_chat(\n", 508 | " manager_nocritic,\n", 509 | " message=\"\"\"\n", 510 | "find papers on LLM applications from arxiv in the last week, create a markdown table of different domains.\n", 511 | "\"\"\",\n", 512 | ")" 513 | ] 514 | } 515 | ], 516 | "metadata": { 517 | "front_matter": { 518 | "tags": ["group chat"], 519 | "description": "Perform research using a group chat with a number of specialized agents" 520 | }, 521 | "kernelspec": { 522 | "display_name": "flaml", 523 | "language": "python", 524 | "name": "python3" 525 | }, 526 | "language_info": { 527 | "codemirror_mode": { 528 | "name": "ipython", 529 | "version": 3 530 | }, 531 | "file_extension": ".py", 532 | "mimetype": "text/x-python", 533 | "name": "python", 534 | "nbconvert_exporter": "python", 535 | "pygments_lexer": "ipython3", 536 | "version": "3.9.17" 537 | }, 538 | "orig_nbformat": 4 539 | }, 540 | "nbformat": 4, 541 | "nbformat_minor": 2 542 | } 543 | -------------------------------------------------------------------------------- /Autogen_planning_with_Agents.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "attachments": {}, 5 | "cell_type": "markdown", 6 | "metadata": {}, 7 | "source": [ 8 | "\"Open" 9 | ] 10 | }, 11 | { 12 | "attachments": {}, 13 | "cell_type": "markdown", 14 | "metadata": { 15 | "slideshow": { 16 | "slide_type": "slide" 17 | } 18 | }, 19 | "source": [ 20 | "# Auto Generated Agent Chat: Collaborative Task Solving with Coding and Planning Agent\n", 21 | "\n", 22 | "AutoGen offers conversable agents powered by LLM, tool, or human, which can be used to perform tasks collectively via automated chat. This framework allows tool use and human participation through multi-agent conversation.\n", 23 | "Please find documentation about this feature [here](https://microsoft.github.io/autogen/docs/Use-Cases/agent_chat).\n", 24 | "\n", 25 | "In this notebook, we demonstrate how to use multiple agents to work together and accomplish a task that requires finding info from the web and coding. `AssistantAgent` is an LLM-based agent that can write and debug Python code (in a Python coding block) for a user to execute for a given task. `UserProxyAgent` is an agent which serves as a proxy for a user to execute the code written by `AssistantAgent`. We further create a planning agent for the assistant agent to consult. The planning agent is a variation of the LLM-based `AssistantAgent` with a different system message.\n", 26 | "\n", 27 | "## Requirements\n", 28 | "\n", 29 | "AutoGen requires `Python>=3.8`. To run this notebook example, please install pyautogen and docker:\n", 30 | "```bash\n", 31 | "pip install pyautogen docker\n", 32 | "```" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": 1, 38 | "metadata": { 39 | "execution": { 40 | "iopub.execute_input": "2023-02-13T23:40:52.317406Z", 41 | "iopub.status.busy": "2023-02-13T23:40:52.316561Z", 42 | "iopub.status.idle": "2023-02-13T23:40:52.321193Z", 43 | "shell.execute_reply": "2023-02-13T23:40:52.320628Z" 44 | } 45 | }, 46 | "outputs": [], 47 | "source": [ 48 | "# %pip install \"pyautogen>=0.2.3\" docker" 49 | ] 50 | }, 51 | { 52 | "attachments": {}, 53 | "cell_type": "markdown", 54 | "metadata": {}, 55 | "source": [ 56 | "## Set your API Endpoint\n", 57 | "\n", 58 | "The [`config_list_from_json`](https://microsoft.github.io/autogen/docs/reference/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file. It first looks for an environment variable with a specified name. The value of the environment variable needs to be a valid json string. If that variable is not found, it looks for a json file with the same name. It filters the configs by filter_dict.\n", 59 | "\n", 60 | "It's OK to have only the OpenAI API key, or only the Azure OpenAI API key + base.\n" 61 | ] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "execution_count": 2, 66 | "metadata": {}, 67 | "outputs": [], 68 | "source": [ 69 | "import autogen\n", 70 | "\n", 71 | "config_list = autogen.config_list_from_json(\n", 72 | " \"OAI_CONFIG_LIST\",\n", 73 | " filter_dict={\n", 74 | " \"model\": [\"gpt-4\", \"gpt-4-0314\", \"gpt4\", \"gpt-4-32k\", \"gpt-4-32k-0314\", \"gpt-4-32k-v0314\"],\n", 75 | " },\n", 76 | ")" 77 | ] 78 | }, 79 | { 80 | "attachments": {}, 81 | "cell_type": "markdown", 82 | "metadata": {}, 83 | "source": [ 84 | "The config list looks like the following:\n", 85 | "```python\n", 86 | "config_list = [\n", 87 | " {\n", 88 | " 'model': 'gpt-4',\n", 89 | " 'api_key': '',\n", 90 | " }, # OpenAI API endpoint for gpt-4\n", 91 | " {\n", 92 | " 'model': 'gpt-4',\n", 93 | " 'api_key': '',\n", 94 | " 'base_url': '',\n", 95 | " 'api_type': 'azure',\n", 96 | " 'api_version': '2024-02-15-preview',\n", 97 | " }, # Azure OpenAI API endpoint for gpt-4\n", 98 | " {\n", 99 | " 'model': 'gpt-4-32k',\n", 100 | " 'api_key': '',\n", 101 | " 'base_url': '',\n", 102 | " 'api_type': 'azure',\n", 103 | " 'api_version': '2024-02-15-preview',\n", 104 | " }, # Azure OpenAI API endpoint for gpt-4-32k\n", 105 | "]\n", 106 | "```\n", 107 | "\n", 108 | "You can set the value of config_list in any way you prefer. Please refer to this [notebook](https://github.com/microsoft/autogen/blob/main/notebook/oai_openai_utils.ipynb) for full code examples of the different methods.\n", 109 | "\n", 110 | "## Construct Agents\n", 111 | "\n", 112 | "We construct the planning agent named \"planner\" and a user proxy agent for the planner named \"planner_user\". We specify `human_input_mode` as \"NEVER\" in the user proxy agent, which will never ask for human feedback. We define `ask_planner` function to send a message to the planner and return the suggestion from the planner." 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 3, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "planner = autogen.AssistantAgent(\n", 122 | " name=\"planner\",\n", 123 | " llm_config={\"config_list\": config_list},\n", 124 | " # the default system message of the AssistantAgent is overwritten here\n", 125 | " system_message=\"You are a helpful AI assistant. You suggest coding and reasoning steps for another AI assistant to accomplish a task. Do not suggest concrete code. For any action beyond writing code or reasoning, convert it to a step that can be implemented by writing code. For example, browsing the web can be implemented by writing code that reads and prints the content of a web page. Finally, inspect the execution result. If the plan is not good, suggest a better plan. If the execution is wrong, analyze the error and suggest a fix.\",\n", 126 | ")\n", 127 | "planner_user = autogen.UserProxyAgent(\n", 128 | " name=\"planner_user\",\n", 129 | " max_consecutive_auto_reply=0, # terminate without auto-reply\n", 130 | " human_input_mode=\"NEVER\",\n", 131 | " code_execution_config={\n", 132 | " \"use_docker\": False\n", 133 | " }, # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n", 134 | ")\n", 135 | "\n", 136 | "\n", 137 | "def ask_planner(message):\n", 138 | " planner_user.initiate_chat(planner, message=message)\n", 139 | " # return the last message received from the planner\n", 140 | " return planner_user.last_message()[\"content\"]" 141 | ] 142 | }, 143 | { 144 | "attachments": {}, 145 | "cell_type": "markdown", 146 | "metadata": {}, 147 | "source": [ 148 | "We construct the assistant agent and the user proxy agent. We specify `human_input_mode` as \"TERMINATE\" in the user proxy agent, which will ask for feedback when it receives a \"TERMINATE\" signal from the assistant agent. We set the `functions` in `AssistantAgent` and `function_map` in `UserProxyAgent` to use the created `ask_planner` function." 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": 4, 154 | "metadata": {}, 155 | "outputs": [], 156 | "source": [ 157 | "# create an AssistantAgent instance named \"assistant\"\n", 158 | "assistant = autogen.AssistantAgent(\n", 159 | " name=\"assistant\",\n", 160 | " llm_config={\n", 161 | " \"temperature\": 0,\n", 162 | " \"timeout\": 600,\n", 163 | " \"cache_seed\": 42,\n", 164 | " \"config_list\": config_list,\n", 165 | " \"functions\": [\n", 166 | " {\n", 167 | " \"name\": \"ask_planner\",\n", 168 | " \"description\": \"ask planner to: 1. get a plan for finishing a task, 2. verify the execution result of the plan and potentially suggest new plan.\",\n", 169 | " \"parameters\": {\n", 170 | " \"type\": \"object\",\n", 171 | " \"properties\": {\n", 172 | " \"message\": {\n", 173 | " \"type\": \"string\",\n", 174 | " \"description\": \"question to ask planner. Make sure the question include enough context, such as the code and the execution result. The planner does not know the conversation between you and the user, unless you share the conversation with the planner.\",\n", 175 | " },\n", 176 | " },\n", 177 | " \"required\": [\"message\"],\n", 178 | " },\n", 179 | " },\n", 180 | " ],\n", 181 | " },\n", 182 | ")\n", 183 | "\n", 184 | "# create a UserProxyAgent instance named \"user_proxy\"\n", 185 | "user_proxy = autogen.UserProxyAgent(\n", 186 | " name=\"user_proxy\",\n", 187 | " human_input_mode=\"TERMINATE\",\n", 188 | " max_consecutive_auto_reply=10,\n", 189 | " # is_termination_msg=lambda x: \"content\" in x and x[\"content\"] is not None and x[\"content\"].rstrip().endswith(\"TERMINATE\"),\n", 190 | " code_execution_config={\n", 191 | " \"work_dir\": \"planning\",\n", 192 | " \"use_docker\": False,\n", 193 | " }, # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n", 194 | " function_map={\"ask_planner\": ask_planner},\n", 195 | ")" 196 | ] 197 | }, 198 | { 199 | "attachments": {}, 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "## Perform a task\n", 204 | "\n", 205 | "We invoke the `initiate_chat()` method of the user proxy agent to start the conversation. When you run the cell below, you will be prompted to provide feedback after the assistant agent sends a \"TERMINATE\" signal at the end of the message. If you don't provide any feedback (by pressing Enter directly), the conversation will finish. Before the \"TERMINATE\" signal, the user proxy agent will try to execute the code suggested by the assistant agent on behalf of the user." 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": 5, 211 | "metadata": {}, 212 | "outputs": [ 213 | { 214 | "name": "stdout", 215 | "output_type": "stream", 216 | "text": [ 217 | "\u001b[33muser_proxy\u001b[0m (to assistant):\n", 218 | "\n", 219 | "Suggest a fix to an open good first issue of flaml\n", 220 | "\n", 221 | "--------------------------------------------------------------------------------\n", 222 | "\u001b[33massistant\u001b[0m (to user_proxy):\n", 223 | "\n", 224 | "To suggest a fix to an open good first issue of FLAML, we first need to fetch the list of open issues labeled as \"good first issue\" from the FLAML GitHub repository. We can do this using the GitHub API. Here is a Python script that fetches and prints the list of open issues labeled as \"good first issue\" from the FLAML repository.\n", 225 | "\n", 226 | "```python\n", 227 | "# filename: fetch_issues.py\n", 228 | "\n", 229 | "import requests\n", 230 | "import json\n", 231 | "\n", 232 | "def fetch_issues():\n", 233 | " url = \"https://api.github.com/repos/microsoft/FLAML/issues\"\n", 234 | " params = {\n", 235 | " \"state\": \"open\",\n", 236 | " \"labels\": \"good first issue\"\n", 237 | " }\n", 238 | " response = requests.get(url, params=params)\n", 239 | " issues = json.loads(response.text)\n", 240 | " for issue in issues:\n", 241 | " print(f\"Issue #{issue['number']}: {issue['title']}\")\n", 242 | "\n", 243 | "if __name__ == \"__main__\":\n", 244 | " fetch_issues()\n", 245 | "```\n", 246 | "\n", 247 | "Please run this script to fetch the list of open issues. After that, I can help you analyze one of the issues and suggest a potential fix.\n", 248 | "\n", 249 | "--------------------------------------------------------------------------------\n", 250 | "\u001b[31m\n", 251 | ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", 252 | "\u001b[31m\n", 253 | ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n" 254 | ] 255 | }, 256 | { 257 | "name": "stderr", 258 | "output_type": "stream", 259 | "text": [ 260 | "execute_code was called without specifying a value for use_docker. Since the python docker package is not available, code will be run natively. Note: this fallback behavior is subject to change\n" 261 | ] 262 | }, 263 | { 264 | "name": "stdout", 265 | "output_type": "stream", 266 | "text": [ 267 | "\u001b[33muser_proxy\u001b[0m (to assistant):\n", 268 | "\n", 269 | "exitcode: 0 (execution succeeded)\n", 270 | "Code output: \n", 271 | "Issue #1228: include that `retrain_full = True` does not include the user provided validation data in the docs.\n", 272 | "Issue #1120: use_label_encoder warning with xgboost\n", 273 | "Issue #981: Running flaml[tune] using \"-O\" flag for python interpreter (optimization - disables assertions) crashes\n", 274 | "Issue #903: Conditional parameter flow2 crash\n", 275 | "Issue #884: indentation space\n", 276 | "Issue #882: Check if openml version is required\n", 277 | "Issue #834: Adjust the indent\n", 278 | "Issue #821: pip install flaml FAIL\n", 279 | "Issue #807: Isolate the ensemble part and expose it to users\n", 280 | "Issue #805: how to pass categorical features names or indices to learner\n", 281 | "Issue #785: Flaml/LightGBM - Shouldn't I found better/faster or equal results from FLAML than direct LightGBM?\n", 282 | "Issue #764: Add an announcement of the discord channel\n", 283 | "Issue #748: Documentation about small budget\n", 284 | "Issue #737: Make zero-shot automl more discoverable\n", 285 | "Issue #509: New HCrystalBall release\n", 286 | "Issue #429: samples about conversion to ONNX\n", 287 | "Issue #413: support anomaly detection\n", 288 | "Issue #304: CatBoost Fails with Keyword 'groups'\n", 289 | "\n", 290 | "\n", 291 | "--------------------------------------------------------------------------------\n", 292 | "\u001b[33massistant\u001b[0m (to user_proxy):\n", 293 | "\n", 294 | "\u001b[32m***** Suggested function Call: ask_planner *****\u001b[0m\n", 295 | "Arguments: \n", 296 | "{\n", 297 | " \"message\": \"Here are the open issues labeled as 'good first issue' in the FLAML repository. Please suggest a plan to fix one of these issues. \\n\\n1. Issue #1228: include that `retrain_full = True` does not include the user provided validation data in the docs.\\n2. Issue #1120: use_label_encoder warning with xgboost\\n3. Issue #981: Running flaml[tune] using \\\"-O\\\" flag for python interpreter (optimization - disables assertions) crashes\\n4. Issue #903: Conditional parameter flow2 crash\\n5. Issue #884: indentation space\\n6. Issue #882: Check if openml version is required\\n7. Issue #834: Adjust the indent\\n8. Issue #821: pip install flaml FAIL\\n9. Issue #807: Isolate the ensemble part and expose it to users\\n10. Issue #805: how to pass categorical features names or indices to learner\\n11. Issue #785: Flaml/LightGBM - Shouldn't I found better/faster or equal results from FLAML than direct LightGBM?\\n12. Issue #764: Add an announcement of the discord channel\\n13. Issue #748: Documentation about small budget\\n14. Issue #737: Make zero-shot automl more discoverable\\n15. Issue #509: New HCrystalBall release\\n16. Issue #429: samples about conversion to ONNX\\n17. Issue #413: support anomaly detection\\n18. Issue #304: CatBoost Fails with Keyword 'groups'\"\n", 298 | "}\n", 299 | "\u001b[32m************************************************\u001b[0m\n", 300 | "\n", 301 | "--------------------------------------------------------------------------------\n", 302 | "\u001b[31m\n", 303 | ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", 304 | "\u001b[35m\n", 305 | ">>>>>>>> EXECUTING FUNCTION ask_planner...\u001b[0m\n", 306 | "\u001b[33mplanner_user\u001b[0m (to planner):\n", 307 | "\n", 308 | "Here are the open issues labeled as 'good first issue' in the FLAML repository. Please suggest a plan to fix one of these issues. \n", 309 | "\n", 310 | "1. Issue #1228: include that `retrain_full = True` does not include the user provided validation data in the docs.\n", 311 | "2. Issue #1120: use_label_encoder warning with xgboost\n", 312 | "3. Issue #981: Running flaml[tune] using \"-O\" flag for python interpreter (optimization - disables assertions) crashes\n", 313 | "4. Issue #903: Conditional parameter flow2 crash\n", 314 | "5. Issue #884: indentation space\n", 315 | "6. Issue #882: Check if openml version is required\n", 316 | "7. Issue #834: Adjust the indent\n", 317 | "8. Issue #821: pip install flaml FAIL\n", 318 | "9. Issue #807: Isolate the ensemble part and expose it to users\n", 319 | "10. Issue #805: how to pass categorical features names or indices to learner\n", 320 | "11. Issue #785: Flaml/LightGBM - Shouldn't I found better/faster or equal results from FLAML than direct LightGBM?\n", 321 | "12. Issue #764: Add an announcement of the discord channel\n", 322 | "13. Issue #748: Documentation about small budget\n", 323 | "14. Issue #737: Make zero-shot automl more discoverable\n", 324 | "15. Issue #509: New HCrystalBall release\n", 325 | "16. Issue #429: samples about conversion to ONNX\n", 326 | "17. Issue #413: support anomaly detection\n", 327 | "18. Issue #304: CatBoost Fails with Keyword 'groups'\n", 328 | "\n", 329 | "--------------------------------------------------------------------------------\n", 330 | "\u001b[33mplanner\u001b[0m (to planner_user):\n", 331 | "\n", 332 | "Here are the steps to fix the first issue (Issue #1228: include that `retrain_full = True` does not include the user provided validation data in the docs):\n", 333 | "\n", 334 | "1. **Identify where the issue is**: Understand the context of `retrain_full = True` within FLAML. Figure out how it works - whether it really ignores the user-provided validation data or not.\n", 335 | "\n", 336 | "2. **Update the documentation**: Based on your understanding, update the corresponding part of the documentation to include a note or clarification about this fact. You can use Markdown language to do the documentation. The note can be written in a clear and concise way.\n", 337 | "\n", 338 | "3. **Commit and Push Code**: After making the necessary changes, commit and push the changes to your repository. Make sure to include a detailed commit message to make it clear what changes were made.\n", 339 | "\n", 340 | "4. **Submit a Pull Request (PR)**: Then submit a pull request to the FLAML repository. In the message of the PR, be sure to reference the issue number (i.e. #1228), to automatically link it.\n", 341 | "\n", 342 | "5. **Wait for Review**: Maintainers will then review your code. They may suggest changes or improvements, and once they're satisfied, they'll merge your changes into the main repository.\n", 343 | "\n", 344 | "6. **Inspect the Results**: After the pull request is merged, verify that the changes have been successfully incorporated and the documentation correctly reflects the behavior of `retrain_full = True`.\n", 345 | "\n", 346 | "Please note, this plan does not involve writing code in any programming language. Instead, it focuses on understanding the library, editing Markdown files, and using Git and GitHub functionalities appropriately. \n", 347 | "\n", 348 | "Should any of your actions result in an error, this could be due to multiple reasons such as misinterpretation of the behaviour of `retrain_full = True`, errors in the markdown syntax, among others. You will need to debug the error based on the specific error message and review your changes. After making corrections, you should commit and push your changes and verify that the error has been fixed.\n", 349 | "\n", 350 | "--------------------------------------------------------------------------------\n", 351 | "\u001b[33muser_proxy\u001b[0m (to assistant):\n", 352 | "\n", 353 | "\u001b[32m***** Response from calling function \"ask_planner\" *****\u001b[0m\n", 354 | "Here are the steps to fix the first issue (Issue #1228: include that `retrain_full = True` does not include the user provided validation data in the docs):\n", 355 | "\n", 356 | "1. **Identify where the issue is**: Understand the context of `retrain_full = True` within FLAML. Figure out how it works - whether it really ignores the user-provided validation data or not.\n", 357 | "\n", 358 | "2. **Update the documentation**: Based on your understanding, update the corresponding part of the documentation to include a note or clarification about this fact. You can use Markdown language to do the documentation. The note can be written in a clear and concise way.\n", 359 | "\n", 360 | "3. **Commit and Push Code**: After making the necessary changes, commit and push the changes to your repository. Make sure to include a detailed commit message to make it clear what changes were made.\n", 361 | "\n", 362 | "4. **Submit a Pull Request (PR)**: Then submit a pull request to the FLAML repository. In the message of the PR, be sure to reference the issue number (i.e. #1228), to automatically link it.\n", 363 | "\n", 364 | "5. **Wait for Review**: Maintainers will then review your code. They may suggest changes or improvements, and once they're satisfied, they'll merge your changes into the main repository.\n", 365 | "\n", 366 | "6. **Inspect the Results**: After the pull request is merged, verify that the changes have been successfully incorporated and the documentation correctly reflects the behavior of `retrain_full = True`.\n", 367 | "\n", 368 | "Please note, this plan does not involve writing code in any programming language. Instead, it focuses on understanding the library, editing Markdown files, and using Git and GitHub functionalities appropriately. \n", 369 | "\n", 370 | "Should any of your actions result in an error, this could be due to multiple reasons such as misinterpretation of the behaviour of `retrain_full = True`, errors in the markdown syntax, among others. You will need to debug the error based on the specific error message and review your changes. After making corrections, you should commit and push your changes and verify that the error has been fixed.\n", 371 | "\u001b[32m********************************************************\u001b[0m\n", 372 | "\n", 373 | "--------------------------------------------------------------------------------\n", 374 | "\u001b[33massistant\u001b[0m (to user_proxy):\n", 375 | "\n", 376 | "To fix the issue #1228, you need to follow these steps:\n", 377 | "\n", 378 | "1. **Identify where the issue is**: Understand the context of `retrain_full = True` within FLAML. Figure out how it works - whether it really ignores the user-provided validation data or not.\n", 379 | "\n", 380 | "2. **Update the documentation**: Based on your understanding, update the corresponding part of the documentation to include a note or clarification about this fact. You can use Markdown language to do the documentation. The note can be written in a clear and concise way.\n", 381 | "\n", 382 | "3. **Commit and Push Code**: After making the necessary changes, commit and push the changes to your repository. Make sure to include a detailed commit message to make it clear what changes were made.\n", 383 | "\n", 384 | "4. **Submit a Pull Request (PR)**: Then submit a pull request to the FLAML repository. In the message of the PR, be sure to reference the issue number (i.e. #1228), to automatically link it.\n", 385 | "\n", 386 | "5. **Wait for Review**: Maintainers will then review your code. They may suggest changes or improvements, and once they're satisfied, they'll merge your changes into the main repository.\n", 387 | "\n", 388 | "6. **Inspect the Results**: After the pull request is merged, verify that the changes have been successfully incorporated and the documentation correctly reflects the behavior of `retrain_full = True`.\n", 389 | "\n", 390 | "Please note, this plan does not involve writing code in any programming language. Instead, it focuses on understanding the library, editing Markdown files, and using Git and GitHub functionalities appropriately. \n", 391 | "\n", 392 | "Should any of your actions result in an error, this could be due to multiple reasons such as misinterpretation of the behaviour of `retrain_full = True`, errors in the markdown syntax, among others. You will need to debug the error based on the specific error message and review your changes. After making corrections, you should commit and push your changes and verify that the error has been fixed.\n", 393 | "\n", 394 | "TERMINATE\n", 395 | "\n", 396 | "--------------------------------------------------------------------------------\n", 397 | "\u001b[31m\n", 398 | ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", 399 | "\u001b[33muser_proxy\u001b[0m (to assistant):\n", 400 | "\n", 401 | "\n", 402 | "\n", 403 | "--------------------------------------------------------------------------------\n", 404 | "\u001b[33massistant\u001b[0m (to user_proxy):\n", 405 | "\n", 406 | "TERMINATE\n", 407 | "\n", 408 | "--------------------------------------------------------------------------------\n", 409 | "\u001b[31m\n", 410 | ">>>>>>>> NO HUMAN INPUT RECEIVED.\u001b[0m\n" 411 | ] 412 | } 413 | ], 414 | "source": [ 415 | "# the assistant receives a message from the user, which contains the task description\n", 416 | "user_proxy.initiate_chat(\n", 417 | " assistant,\n", 418 | " message=\"\"\"Suggest a fix to an open good first issue of flaml\"\"\",\n", 419 | ")" 420 | ] 421 | }, 422 | { 423 | "attachments": {}, 424 | "cell_type": "markdown", 425 | "metadata": {}, 426 | "source": [ 427 | "When the assistant needs to consult the planner, it suggests a function call to `ask_planner`. When this happens, a line like the following will be displayed:\n", 428 | "\n", 429 | "***** Suggested function Call: ask_planner *****\n" 430 | ] 431 | } 432 | ], 433 | "metadata": { 434 | "kernelspec": { 435 | "display_name": "Python 3", 436 | "language": "python", 437 | "name": "python3" 438 | }, 439 | "language_info": { 440 | "codemirror_mode": { 441 | "name": "ipython", 442 | "version": 3 443 | }, 444 | "file_extension": ".py", 445 | "mimetype": "text/x-python", 446 | "name": "python", 447 | "nbconvert_exporter": "python", 448 | "pygments_lexer": "ipython3", 449 | "version": "3.11.4" 450 | }, 451 | "vscode": { 452 | "interpreter": { 453 | "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" 454 | } 455 | }, 456 | "widgets": { 457 | "application/vnd.jupyter.widget-state+json": { 458 | "state": { 459 | "2d910cfd2d2a4fc49fc30fbbdc5576a7": { 460 | "model_module": "@jupyter-widgets/base", 461 | "model_module_version": "2.0.0", 462 | "model_name": "LayoutModel", 463 | "state": { 464 | "_model_module": "@jupyter-widgets/base", 465 | "_model_module_version": "2.0.0", 466 | "_model_name": "LayoutModel", 467 | "_view_count": null, 468 | "_view_module": "@jupyter-widgets/base", 469 | "_view_module_version": "2.0.0", 470 | "_view_name": "LayoutView", 471 | "align_content": null, 472 | "align_items": null, 473 | "align_self": null, 474 | "border_bottom": null, 475 | "border_left": null, 476 | "border_right": null, 477 | "border_top": null, 478 | "bottom": null, 479 | "display": null, 480 | "flex": null, 481 | "flex_flow": null, 482 | "grid_area": null, 483 | "grid_auto_columns": null, 484 | "grid_auto_flow": null, 485 | "grid_auto_rows": null, 486 | "grid_column": null, 487 | "grid_gap": null, 488 | "grid_row": null, 489 | "grid_template_areas": null, 490 | "grid_template_columns": null, 491 | "grid_template_rows": null, 492 | "height": null, 493 | "justify_content": null, 494 | "justify_items": null, 495 | "left": null, 496 | "margin": null, 497 | "max_height": null, 498 | "max_width": null, 499 | "min_height": null, 500 | "min_width": null, 501 | "object_fit": null, 502 | "object_position": null, 503 | "order": null, 504 | "overflow": null, 505 | "padding": null, 506 | "right": null, 507 | "top": null, 508 | "visibility": null, 509 | "width": null 510 | } 511 | }, 512 | "454146d0f7224f038689031002906e6f": { 513 | "model_module": "@jupyter-widgets/controls", 514 | "model_module_version": "2.0.0", 515 | "model_name": "HBoxModel", 516 | "state": { 517 | "_dom_classes": [], 518 | "_model_module": "@jupyter-widgets/controls", 519 | "_model_module_version": "2.0.0", 520 | "_model_name": "HBoxModel", 521 | "_view_count": null, 522 | "_view_module": "@jupyter-widgets/controls", 523 | "_view_module_version": "2.0.0", 524 | "_view_name": "HBoxView", 525 | "box_style": "", 526 | "children": [ 527 | "IPY_MODEL_e4ae2b6f5a974fd4bafb6abb9d12ff26", 528 | "IPY_MODEL_577e1e3cc4db4942b0883577b3b52755", 529 | "IPY_MODEL_b40bdfb1ac1d4cffb7cefcb870c64d45" 530 | ], 531 | "layout": "IPY_MODEL_dc83c7bff2f241309537a8119dfc7555", 532 | "tabbable": null, 533 | "tooltip": null 534 | } 535 | }, 536 | "577e1e3cc4db4942b0883577b3b52755": { 537 | "model_module": "@jupyter-widgets/controls", 538 | "model_module_version": "2.0.0", 539 | "model_name": "FloatProgressModel", 540 | "state": { 541 | "_dom_classes": [], 542 | "_model_module": "@jupyter-widgets/controls", 543 | "_model_module_version": "2.0.0", 544 | "_model_name": "FloatProgressModel", 545 | "_view_count": null, 546 | "_view_module": "@jupyter-widgets/controls", 547 | "_view_module_version": "2.0.0", 548 | "_view_name": "ProgressView", 549 | "bar_style": "success", 550 | "description": "", 551 | "description_allow_html": false, 552 | "layout": "IPY_MODEL_2d910cfd2d2a4fc49fc30fbbdc5576a7", 553 | "max": 1, 554 | "min": 0, 555 | "orientation": "horizontal", 556 | "style": "IPY_MODEL_74a6ba0c3cbc4051be0a83e152fe1e62", 557 | "tabbable": null, 558 | "tooltip": null, 559 | "value": 1 560 | } 561 | }, 562 | "6086462a12d54bafa59d3c4566f06cb2": { 563 | "model_module": "@jupyter-widgets/base", 564 | "model_module_version": "2.0.0", 565 | "model_name": "LayoutModel", 566 | "state": { 567 | "_model_module": "@jupyter-widgets/base", 568 | "_model_module_version": "2.0.0", 569 | "_model_name": "LayoutModel", 570 | "_view_count": null, 571 | "_view_module": "@jupyter-widgets/base", 572 | "_view_module_version": "2.0.0", 573 | "_view_name": "LayoutView", 574 | "align_content": null, 575 | "align_items": null, 576 | "align_self": null, 577 | "border_bottom": null, 578 | "border_left": null, 579 | "border_right": null, 580 | "border_top": null, 581 | "bottom": null, 582 | "display": null, 583 | "flex": null, 584 | "flex_flow": null, 585 | "grid_area": null, 586 | "grid_auto_columns": null, 587 | "grid_auto_flow": null, 588 | "grid_auto_rows": null, 589 | "grid_column": null, 590 | "grid_gap": null, 591 | "grid_row": null, 592 | "grid_template_areas": null, 593 | "grid_template_columns": null, 594 | "grid_template_rows": null, 595 | "height": null, 596 | "justify_content": null, 597 | "justify_items": null, 598 | "left": null, 599 | "margin": null, 600 | "max_height": null, 601 | "max_width": null, 602 | "min_height": null, 603 | "min_width": null, 604 | "object_fit": null, 605 | "object_position": null, 606 | "order": null, 607 | "overflow": null, 608 | "padding": null, 609 | "right": null, 610 | "top": null, 611 | "visibility": null, 612 | "width": null 613 | } 614 | }, 615 | "74a6ba0c3cbc4051be0a83e152fe1e62": { 616 | "model_module": "@jupyter-widgets/controls", 617 | "model_module_version": "2.0.0", 618 | "model_name": "ProgressStyleModel", 619 | "state": { 620 | "_model_module": "@jupyter-widgets/controls", 621 | "_model_module_version": "2.0.0", 622 | "_model_name": "ProgressStyleModel", 623 | "_view_count": null, 624 | "_view_module": "@jupyter-widgets/base", 625 | "_view_module_version": "2.0.0", 626 | "_view_name": "StyleView", 627 | "bar_color": null, 628 | "description_width": "" 629 | } 630 | }, 631 | "7d3f3d9e15894d05a4d188ff4f466554": { 632 | "model_module": "@jupyter-widgets/controls", 633 | "model_module_version": "2.0.0", 634 | "model_name": "HTMLStyleModel", 635 | "state": { 636 | "_model_module": "@jupyter-widgets/controls", 637 | "_model_module_version": "2.0.0", 638 | "_model_name": "HTMLStyleModel", 639 | "_view_count": null, 640 | "_view_module": "@jupyter-widgets/base", 641 | "_view_module_version": "2.0.0", 642 | "_view_name": "StyleView", 643 | "background": null, 644 | "description_width": "", 645 | "font_size": null, 646 | "text_color": null 647 | } 648 | }, 649 | "b40bdfb1ac1d4cffb7cefcb870c64d45": { 650 | "model_module": "@jupyter-widgets/controls", 651 | "model_module_version": "2.0.0", 652 | "model_name": "HTMLModel", 653 | "state": { 654 | "_dom_classes": [], 655 | "_model_module": "@jupyter-widgets/controls", 656 | "_model_module_version": "2.0.0", 657 | "_model_name": "HTMLModel", 658 | "_view_count": null, 659 | "_view_module": "@jupyter-widgets/controls", 660 | "_view_module_version": "2.0.0", 661 | "_view_name": "HTMLView", 662 | "description": "", 663 | "description_allow_html": false, 664 | "layout": "IPY_MODEL_f1355871cc6f4dd4b50d9df5af20e5c8", 665 | "placeholder": "​", 666 | "style": "IPY_MODEL_ca245376fd9f4354af6b2befe4af4466", 667 | "tabbable": null, 668 | "tooltip": null, 669 | "value": " 1/1 [00:00<00:00, 44.69it/s]" 670 | } 671 | }, 672 | "ca245376fd9f4354af6b2befe4af4466": { 673 | "model_module": "@jupyter-widgets/controls", 674 | "model_module_version": "2.0.0", 675 | "model_name": "HTMLStyleModel", 676 | "state": { 677 | "_model_module": "@jupyter-widgets/controls", 678 | "_model_module_version": "2.0.0", 679 | "_model_name": "HTMLStyleModel", 680 | "_view_count": null, 681 | "_view_module": "@jupyter-widgets/base", 682 | "_view_module_version": "2.0.0", 683 | "_view_name": "StyleView", 684 | "background": null, 685 | "description_width": "", 686 | "font_size": null, 687 | "text_color": null 688 | } 689 | }, 690 | "dc83c7bff2f241309537a8119dfc7555": { 691 | "model_module": "@jupyter-widgets/base", 692 | "model_module_version": "2.0.0", 693 | "model_name": "LayoutModel", 694 | "state": { 695 | "_model_module": "@jupyter-widgets/base", 696 | "_model_module_version": "2.0.0", 697 | "_model_name": "LayoutModel", 698 | "_view_count": null, 699 | "_view_module": "@jupyter-widgets/base", 700 | "_view_module_version": "2.0.0", 701 | "_view_name": "LayoutView", 702 | "align_content": null, 703 | "align_items": null, 704 | "align_self": null, 705 | "border_bottom": null, 706 | "border_left": null, 707 | "border_right": null, 708 | "border_top": null, 709 | "bottom": null, 710 | "display": null, 711 | "flex": null, 712 | "flex_flow": null, 713 | "grid_area": null, 714 | "grid_auto_columns": null, 715 | "grid_auto_flow": null, 716 | "grid_auto_rows": null, 717 | "grid_column": null, 718 | "grid_gap": null, 719 | "grid_row": null, 720 | "grid_template_areas": null, 721 | "grid_template_columns": null, 722 | "grid_template_rows": null, 723 | "height": null, 724 | "justify_content": null, 725 | "justify_items": null, 726 | "left": null, 727 | "margin": null, 728 | "max_height": null, 729 | "max_width": null, 730 | "min_height": null, 731 | "min_width": null, 732 | "object_fit": null, 733 | "object_position": null, 734 | "order": null, 735 | "overflow": null, 736 | "padding": null, 737 | "right": null, 738 | "top": null, 739 | "visibility": null, 740 | "width": null 741 | } 742 | }, 743 | "e4ae2b6f5a974fd4bafb6abb9d12ff26": { 744 | "model_module": "@jupyter-widgets/controls", 745 | "model_module_version": "2.0.0", 746 | "model_name": "HTMLModel", 747 | "state": { 748 | "_dom_classes": [], 749 | "_model_module": "@jupyter-widgets/controls", 750 | "_model_module_version": "2.0.0", 751 | "_model_name": "HTMLModel", 752 | "_view_count": null, 753 | "_view_module": "@jupyter-widgets/controls", 754 | "_view_module_version": "2.0.0", 755 | "_view_name": "HTMLView", 756 | "description": "", 757 | "description_allow_html": false, 758 | "layout": "IPY_MODEL_6086462a12d54bafa59d3c4566f06cb2", 759 | "placeholder": "​", 760 | "style": "IPY_MODEL_7d3f3d9e15894d05a4d188ff4f466554", 761 | "tabbable": null, 762 | "tooltip": null, 763 | "value": "100%" 764 | } 765 | }, 766 | "f1355871cc6f4dd4b50d9df5af20e5c8": { 767 | "model_module": "@jupyter-widgets/base", 768 | "model_module_version": "2.0.0", 769 | "model_name": "LayoutModel", 770 | "state": { 771 | "_model_module": "@jupyter-widgets/base", 772 | "_model_module_version": "2.0.0", 773 | "_model_name": "LayoutModel", 774 | "_view_count": null, 775 | "_view_module": "@jupyter-widgets/base", 776 | "_view_module_version": "2.0.0", 777 | "_view_name": "LayoutView", 778 | "align_content": null, 779 | "align_items": null, 780 | "align_self": null, 781 | "border_bottom": null, 782 | "border_left": null, 783 | "border_right": null, 784 | "border_top": null, 785 | "bottom": null, 786 | "display": null, 787 | "flex": null, 788 | "flex_flow": null, 789 | "grid_area": null, 790 | "grid_auto_columns": null, 791 | "grid_auto_flow": null, 792 | "grid_auto_rows": null, 793 | "grid_column": null, 794 | "grid_gap": null, 795 | "grid_row": null, 796 | "grid_template_areas": null, 797 | "grid_template_columns": null, 798 | "grid_template_rows": null, 799 | "height": null, 800 | "justify_content": null, 801 | "justify_items": null, 802 | "left": null, 803 | "margin": null, 804 | "max_height": null, 805 | "max_width": null, 806 | "min_height": null, 807 | "min_width": null, 808 | "object_fit": null, 809 | "object_position": null, 810 | "order": null, 811 | "overflow": null, 812 | "padding": null, 813 | "right": null, 814 | "top": null, 815 | "visibility": null, 816 | "width": null 817 | } 818 | } 819 | }, 820 | "version_major": 2, 821 | "version_minor": 0 822 | } 823 | } 824 | }, 825 | "nbformat": 4, 826 | "nbformat_minor": 2 827 | } 828 | -------------------------------------------------------------------------------- /Autogen_groupchat_RAG.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "attachments": {}, 5 | "cell_type": "markdown", 6 | "metadata": {}, 7 | "source": [ 8 | "# Group Chat with Retrieval Augmented Generation\n", 9 | "\n", 10 | "AutoGen supports conversable agents powered by LLMs, tools, or humans, performing tasks collectively via automated chat. This framework allows tool use and human participation through multi-agent conversation.\n", 11 | "Please find documentation about this feature [here](https://microsoft.github.io/autogen/docs/Use-Cases/agent_chat).\n", 12 | "\n", 13 | "````{=mdx}\n", 14 | ":::info Requirements\n", 15 | "Some extra dependencies are needed for this notebook, which can be installed via pip:\n", 16 | "\n", 17 | "```bash\n", 18 | "pip install pyautogen[retrievechat]\n", 19 | "```\n", 20 | "\n", 21 | "For more information, please refer to the [installation guide](/docs/installation/).\n", 22 | ":::\n", 23 | "````" 24 | ] 25 | }, 26 | { 27 | "attachments": {}, 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "## Set your API Endpoint\n", 32 | "\n", 33 | "The [`config_list_from_json`](https://microsoft.github.io/autogen/docs/reference/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file." 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": 1, 39 | "metadata": {}, 40 | "outputs": [ 41 | { 42 | "name": "stdout", 43 | "output_type": "stream", 44 | "text": [ 45 | "LLM models: ['gpt4-1106-preview', 'gpt-35-turbo', 'gpt-35-turbo-0613']\n" 46 | ] 47 | } 48 | ], 49 | "source": [ 50 | "import chromadb\n", 51 | "from typing_extensions import Annotated\n", 52 | "\n", 53 | "import autogen\n", 54 | "from autogen import AssistantAgent\n", 55 | "from autogen.agentchat.contrib.retrieve_user_proxy_agent import RetrieveUserProxyAgent\n", 56 | "\n", 57 | "config_list = autogen.config_list_from_json(\"OAI_CONFIG_LIST\")\n", 58 | "\n", 59 | "print(\"LLM models: \", [config_list[i][\"model\"] for i in range(len(config_list))])" 60 | ] 61 | }, 62 | { 63 | "attachments": {}, 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "````{=mdx}\n", 68 | ":::tip\n", 69 | "Learn more about configuring LLMs for agents [here](/docs/topics/llm_configuration).\n", 70 | ":::\n", 71 | "````\n", 72 | "\n", 73 | "## Construct Agents" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": 2, 79 | "metadata": {}, 80 | "outputs": [ 81 | { 82 | "name": "stderr", 83 | "output_type": "stream", 84 | "text": [ 85 | "/home/lijiang1/anaconda3/envs/autogen/lib/python3.10/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n", 86 | " torch.utils._pytree._register_pytree_node(\n" 87 | ] 88 | } 89 | ], 90 | "source": [ 91 | "def termination_msg(x):\n", 92 | " return isinstance(x, dict) and \"TERMINATE\" == str(x.get(\"content\", \"\"))[-9:].upper()\n", 93 | "\n", 94 | "\n", 95 | "llm_config = {\"config_list\": config_list, \"timeout\": 60, \"temperature\": 0.8, \"seed\": 1234}\n", 96 | "\n", 97 | "boss = autogen.UserProxyAgent(\n", 98 | " name=\"Boss\",\n", 99 | " is_termination_msg=termination_msg,\n", 100 | " human_input_mode=\"NEVER\",\n", 101 | " code_execution_config=False, # we don't want to execute code in this case.\n", 102 | " default_auto_reply=\"Reply `TERMINATE` if the task is done.\",\n", 103 | " description=\"The boss who ask questions and give tasks.\",\n", 104 | ")\n", 105 | "\n", 106 | "boss_aid = RetrieveUserProxyAgent(\n", 107 | " name=\"Boss_Assistant\",\n", 108 | " is_termination_msg=termination_msg,\n", 109 | " human_input_mode=\"NEVER\",\n", 110 | " default_auto_reply=\"Reply `TERMINATE` if the task is done.\",\n", 111 | " max_consecutive_auto_reply=3,\n", 112 | " retrieve_config={\n", 113 | " \"task\": \"code\",\n", 114 | " \"docs_path\": \"https://raw.githubusercontent.com/microsoft/FLAML/main/website/docs/Examples/Integrate%20-%20Spark.md\",\n", 115 | " \"chunk_token_size\": 1000,\n", 116 | " \"model\": config_list[0][\"model\"],\n", 117 | " \"collection_name\": \"groupchat\",\n", 118 | " \"get_or_create\": True,\n", 119 | " },\n", 120 | " code_execution_config=False, # we don't want to execute code in this case.\n", 121 | " description=\"Assistant who has extra content retrieval power for solving difficult problems.\",\n", 122 | ")\n", 123 | "\n", 124 | "coder = AssistantAgent(\n", 125 | " name=\"Senior_Python_Engineer\",\n", 126 | " is_termination_msg=termination_msg,\n", 127 | " system_message=\"You are a senior python engineer, you provide python code to answer questions. Reply `TERMINATE` in the end when everything is done.\",\n", 128 | " llm_config=llm_config,\n", 129 | " description=\"Senior Python Engineer who can write code to solve problems and answer questions.\",\n", 130 | ")\n", 131 | "\n", 132 | "pm = autogen.AssistantAgent(\n", 133 | " name=\"Product_Manager\",\n", 134 | " is_termination_msg=termination_msg,\n", 135 | " system_message=\"You are a product manager. Reply `TERMINATE` in the end when everything is done.\",\n", 136 | " llm_config=llm_config,\n", 137 | " description=\"Product Manager who can design and plan the project.\",\n", 138 | ")\n", 139 | "\n", 140 | "reviewer = autogen.AssistantAgent(\n", 141 | " name=\"Code_Reviewer\",\n", 142 | " is_termination_msg=termination_msg,\n", 143 | " system_message=\"You are a code reviewer. Reply `TERMINATE` in the end when everything is done.\",\n", 144 | " llm_config=llm_config,\n", 145 | " description=\"Code Reviewer who can review the code.\",\n", 146 | ")\n", 147 | "\n", 148 | "PROBLEM = \"How to use spark for parallel training in FLAML? Give me sample code.\"\n", 149 | "\n", 150 | "\n", 151 | "def _reset_agents():\n", 152 | " boss.reset()\n", 153 | " boss_aid.reset()\n", 154 | " coder.reset()\n", 155 | " pm.reset()\n", 156 | " reviewer.reset()\n", 157 | "\n", 158 | "\n", 159 | "def rag_chat():\n", 160 | " _reset_agents()\n", 161 | " groupchat = autogen.GroupChat(\n", 162 | " agents=[boss_aid, pm, coder, reviewer], messages=[], max_round=12, speaker_selection_method=\"round_robin\"\n", 163 | " )\n", 164 | " manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config)\n", 165 | "\n", 166 | " # Start chatting with boss_aid as this is the user proxy agent.\n", 167 | " boss_aid.initiate_chat(\n", 168 | " manager,\n", 169 | " message=boss_aid.message_generator,\n", 170 | " problem=PROBLEM,\n", 171 | " n_results=3,\n", 172 | " )\n", 173 | "\n", 174 | "\n", 175 | "def norag_chat():\n", 176 | " _reset_agents()\n", 177 | " groupchat = autogen.GroupChat(\n", 178 | " agents=[boss, pm, coder, reviewer],\n", 179 | " messages=[],\n", 180 | " max_round=12,\n", 181 | " speaker_selection_method=\"auto\",\n", 182 | " allow_repeat_speaker=False,\n", 183 | " )\n", 184 | " manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config)\n", 185 | "\n", 186 | " # Start chatting with the boss as this is the user proxy agent.\n", 187 | " boss.initiate_chat(\n", 188 | " manager,\n", 189 | " message=PROBLEM,\n", 190 | " )\n", 191 | "\n", 192 | "\n", 193 | "def call_rag_chat():\n", 194 | " _reset_agents()\n", 195 | "\n", 196 | " # In this case, we will have multiple user proxy agents and we don't initiate the chat\n", 197 | " # with RAG user proxy agent.\n", 198 | " # In order to use RAG user proxy agent, we need to wrap RAG agents in a function and call\n", 199 | " # it from other agents.\n", 200 | " def retrieve_content(\n", 201 | " message: Annotated[\n", 202 | " str,\n", 203 | " \"Refined message which keeps the original meaning and can be used to retrieve content for code generation and question answering.\",\n", 204 | " ],\n", 205 | " n_results: Annotated[int, \"number of results\"] = 3,\n", 206 | " ) -> str:\n", 207 | " boss_aid.n_results = n_results # Set the number of results to be retrieved.\n", 208 | " # Check if we need to update the context.\n", 209 | " update_context_case1, update_context_case2 = boss_aid._check_update_context(message)\n", 210 | " if (update_context_case1 or update_context_case2) and boss_aid.update_context:\n", 211 | " boss_aid.problem = message if not hasattr(boss_aid, \"problem\") else boss_aid.problem\n", 212 | " _, ret_msg = boss_aid._generate_retrieve_user_reply(message)\n", 213 | " else:\n", 214 | " _context = {\"problem\": message, \"n_results\": n_results}\n", 215 | " ret_msg = boss_aid.message_generator(boss_aid, None, _context)\n", 216 | " return ret_msg if ret_msg else message\n", 217 | "\n", 218 | " boss_aid.human_input_mode = \"NEVER\" # Disable human input for boss_aid since it only retrieves content.\n", 219 | "\n", 220 | " for caller in [pm, coder, reviewer]:\n", 221 | " d_retrieve_content = caller.register_for_llm(\n", 222 | " description=\"retrieve content for code generation and question answering.\", api_style=\"function\"\n", 223 | " )(retrieve_content)\n", 224 | "\n", 225 | " for executor in [boss, pm]:\n", 226 | " executor.register_for_execution()(d_retrieve_content)\n", 227 | "\n", 228 | " groupchat = autogen.GroupChat(\n", 229 | " agents=[boss, pm, coder, reviewer],\n", 230 | " messages=[],\n", 231 | " max_round=12,\n", 232 | " speaker_selection_method=\"round_robin\",\n", 233 | " allow_repeat_speaker=False,\n", 234 | " )\n", 235 | "\n", 236 | " manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config)\n", 237 | "\n", 238 | " # Start chatting with the boss as this is the user proxy agent.\n", 239 | " boss.initiate_chat(\n", 240 | " manager,\n", 241 | " message=PROBLEM,\n", 242 | " )" 243 | ] 244 | }, 245 | { 246 | "attachments": {}, 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "## Start Chat\n", 251 | "\n", 252 | "### UserProxyAgent doesn't get the correct code\n", 253 | "[FLAML](https://github.com/microsoft/FLAML) was open sourced in 2020, so ChatGPT is familiar with it. However, Spark-related APIs were added in 2022, so they were not in ChatGPT's training data. As a result, we end up with invalid code." 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": 3, 259 | "metadata": {}, 260 | "outputs": [ 261 | { 262 | "name": "stdout", 263 | "output_type": "stream", 264 | "text": [ 265 | "\u001b[33mBoss\u001b[0m (to chat_manager):\n", 266 | "\n", 267 | "How to use spark for parallel training in FLAML? Give me sample code.\n", 268 | "\n", 269 | "--------------------------------------------------------------------------------\n" 270 | ] 271 | }, 272 | { 273 | "name": "stdout", 274 | "output_type": "stream", 275 | "text": [ 276 | "\u001b[33mSenior_Python_Engineer\u001b[0m (to chat_manager):\n", 277 | "\n", 278 | "To use Spark for parallel training in FLAML (Fast and Lightweight AutoML), you would need to set up a Spark cluster and utilize the `spark` backend for joblib, which FLAML uses internally for parallel training. Here’s an example of how you might set up and use Spark with FLAML for AutoML tasks:\n", 279 | "\n", 280 | "Firstly, ensure that you have the Spark cluster set up and the `pyspark` and `joblib-spark` packages installed in your environment. You can install the required packages using pip if they are not already installed:\n", 281 | "\n", 282 | "```python\n", 283 | "!pip install flaml pyspark joblib-spark\n", 284 | "```\n", 285 | "\n", 286 | "Here's a sample code snippet that demonstrates how to use FLAML with Spark for parallel training:\n", 287 | "\n", 288 | "```python\n", 289 | "from flaml import AutoML\n", 290 | "from pyspark.sql import SparkSession\n", 291 | "from sklearn.datasets import load_digits\n", 292 | "from joblibspark import register_spark\n", 293 | "\n", 294 | "# Initialize a Spark session\n", 295 | "spark = SparkSession.builder \\\n", 296 | " .master(\"local[*]\") \\\n", 297 | " .appName(\"FLAML_Spark_Example\") \\\n", 298 | " .getOrCreate()\n", 299 | "\n", 300 | "# Register the joblib spark backend\n", 301 | "register_spark() # This registers the backend for parallel processing\n", 302 | "\n", 303 | "# Load sample data\n", 304 | "X, y = load_digits(return_X_y=True)\n", 305 | "\n", 306 | "# Initialize an AutoML instance\n", 307 | "automl = AutoML()\n", 308 | "\n", 309 | "# Define the settings for the AutoML run\n", 310 | "settings = {\n", 311 | " \"time_budget\": 60, # Total running time in seconds\n", 312 | " \"metric\": 'accuracy', # Primary metric for evaluation\n", 313 | " \"task\": 'classification', # Task type\n", 314 | " \"n_jobs\": -1, # Number of jobs to run in parallel (use -1 for all)\n", 315 | " \"estimator_list\": ['lgbm', 'rf', 'xgboost'], # List of estimators to consider\n", 316 | " \"log_file_name\": \"flaml_log.txt\", # Log file name\n", 317 | "}\n", 318 | "\n", 319 | "# Run the AutoML search with Spark backend\n", 320 | "automl.fit(X_train=X, y_train=y, **settings)\n", 321 | "\n", 322 | "# Output the best model and its performance\n", 323 | "print(f\"Best ML model: {automl.model}\")\n", 324 | "print(f\"Best ML model's accuracy: {automl.best_loss}\")\n", 325 | "\n", 326 | "# Stop the Spark session\n", 327 | "spark.stop()\n", 328 | "```\n", 329 | "\n", 330 | "The `register_spark()` function from `joblib-spark` is used to register the Spark backend with joblib, which is utilized for parallel training within FLAML. The `n_jobs=-1` parameter tells FLAML to use all available Spark executors for parallel training.\n", 331 | "\n", 332 | "Please note that the actual process of setting up a Spark cluster can be complex and might involve additional steps such as configuring Spark workers, allocating resources, and more, which are beyond the scope of this code snippet.\n", 333 | "\n", 334 | "If you encounter any issues or need to adjust configurations for your specific Spark setup, please refer to the Spark and FLAML documentation for more details.\n", 335 | "\n", 336 | "When you run the code, ensure that your Spark cluster is properly configured and accessible from your Python environment. Adjust the `.master(\"local[*]\")` to point to your Spark master's URL if you are running a cluster that is not local.\n", 337 | "\n", 338 | "--------------------------------------------------------------------------------\n", 339 | "To use Spark for parallel training in FLAML (Fast and Lightweight AutoML), you would need to set up a Spark cluster and utilize the `spark` backend for joblib, which FLAML uses internally for parallel training. Here’s an example of how you might set up and use Spark with FLAML for AutoML tasks:\n", 340 | "\n", 341 | "Firstly, ensure that you have the Spark cluster set up and the `pyspark` and `joblib-spark` packages installed in your environment. You can install the required packages using pip if they are not already installed:\n", 342 | "\n", 343 | "```python\n", 344 | "!pip install flaml pyspark joblib-spark\n", 345 | "```\n", 346 | "\n", 347 | "Here's a sample code snippet that demonstrates how to use FLAML with Spark for parallel training:\n", 348 | "\n", 349 | "```python\n", 350 | "from flaml import AutoML\n", 351 | "from pyspark.sql import SparkSession\n", 352 | "from sklearn.datasets import load_digits\n", 353 | "from joblibspark import register_spark\n", 354 | "\n", 355 | "# Initialize a Spark session\n", 356 | "spark = SparkSession.builder \\\n", 357 | " .master(\"local[*]\") \\\n", 358 | " .appName(\"FLAML_Spark_Example\") \\\n", 359 | " .getOrCreate()\n", 360 | "\n", 361 | "# Register the joblib spark backend\n", 362 | "register_spark() # This registers the backend for parallel processing\n", 363 | "\n", 364 | "# Load sample data\n", 365 | "X, y = load_digits(return_X_y=True)\n", 366 | "\n", 367 | "# Initialize an AutoML instance\n", 368 | "automl = AutoML()\n", 369 | "\n", 370 | "# Define the settings for the AutoML run\n", 371 | "settings = {\n", 372 | " \"time_budget\": 60, # Total running time in seconds\n", 373 | " \"metric\": 'accuracy', # Primary metric for evaluation\n", 374 | " \"task\": 'classification', # Task type\n", 375 | " \"n_jobs\": -1, # Number of jobs to run in parallel (use -1 for all)\n", 376 | " \"estimator_list\": ['lgbm', 'rf', 'xgboost'], # List of estimators to consider\n", 377 | " \"log_file_name\": \"flaml_log.txt\", # Log file name\n", 378 | "}\n", 379 | "\n", 380 | "# Run the AutoML search with Spark backend\n", 381 | "automl.fit(X_train=X, y_train=y, **settings)\n", 382 | "\n", 383 | "# Output the best model and its performance\n", 384 | "print(f\"Best ML model: {automl.model}\")\n", 385 | "print(f\"Best ML model's accuracy: {automl.best_loss}\")\n", 386 | "\n", 387 | "# Stop the Spark session\n", 388 | "spark.stop()\n", 389 | "```\n", 390 | "\n", 391 | "The `register_spark()` function from `joblib-spark` is used to register the Spark backend with joblib, which is utilized for parallel training within FLAML. The `n_jobs=-1` parameter tells FLAML to use all available Spark executors for parallel training.\n", 392 | "\n", 393 | "Please note that the actual process of setting up a Spark cluster can be complex and might involve additional steps such as configuring Spark workers, allocating resources, and more, which are beyond the scope of this code snippet.\n", 394 | "\n", 395 | "If you encounter any issues or need to adjust configurations for your specific Spark setup, please refer to the Spark and FLAML documentation for more details.\n", 396 | "\n", 397 | "When you run the code, ensure that your Spark cluster is properly configured and accessible from your Python environment. Adjust the `.master(\"local[*]\")` to point to your Spark master's URL if you are running a cluster that is not local.\n", 398 | "\n", 399 | "--------------------------------------------------------------------------------\n", 400 | "\u001b[33mCode_Reviewer\u001b[0m (to chat_manager):\n", 401 | "\n", 402 | "TERMINATE\n", 403 | "\n", 404 | "--------------------------------------------------------------------------------\n" 405 | ] 406 | } 407 | ], 408 | "source": [ 409 | "norag_chat()" 410 | ] 411 | }, 412 | { 413 | "attachments": {}, 414 | "cell_type": "markdown", 415 | "metadata": {}, 416 | "source": [ 417 | "### RetrieveUserProxyAgent get the correct code\n", 418 | "Since RetrieveUserProxyAgent can perform retrieval-augmented generation based on the given documentation file, ChatGPT can generate the correct code for us!" 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": 4, 424 | "metadata": {}, 425 | "outputs": [ 426 | { 427 | "name": "stderr", 428 | "output_type": "stream", 429 | "text": [ 430 | "2024-04-07 18:26:04,562 - autogen.agentchat.contrib.retrieve_user_proxy_agent - INFO - \u001b[32mUse the existing collection `groupchat`.\u001b[0m\n" 431 | ] 432 | }, 433 | { 434 | "name": "stdout", 435 | "output_type": "stream", 436 | "text": [ 437 | "Trying to create collection.\n" 438 | ] 439 | }, 440 | { 441 | "name": "stderr", 442 | "output_type": "stream", 443 | "text": [ 444 | "2024-04-07 18:26:05,485 - autogen.agentchat.contrib.retrieve_user_proxy_agent - INFO - Found 1 chunks.\u001b[0m\n", 445 | "Number of requested results 3 is greater than number of elements in index 1, updating n_results = 1\n", 446 | "Model gpt4-1106-preview not found. Using cl100k_base encoding.\n" 447 | ] 448 | }, 449 | { 450 | "name": "stdout", 451 | "output_type": "stream", 452 | "text": [ 453 | "VectorDB returns doc_ids: [['bdfbc921']]\n", 454 | "\u001b[32mAdding content of doc bdfbc921 to context.\u001b[0m\n", 455 | "\u001b[33mBoss_Assistant\u001b[0m (to chat_manager):\n", 456 | "\n", 457 | "You're a retrieve augmented coding assistant. You answer user's questions based on your own knowledge and the\n", 458 | "context provided by the user.\n", 459 | "If you can't answer the question with or without the current context, you should reply exactly `UPDATE CONTEXT`.\n", 460 | "For code generation, you must obey the following rules:\n", 461 | "Rule 1. You MUST NOT install any packages because all the packages needed are already installed.\n", 462 | "Rule 2. You must follow the formats below to write your code:\n", 463 | "```language\n", 464 | "# your code\n", 465 | "```\n", 466 | "\n", 467 | "User's question is: How to use spark for parallel training in FLAML? Give me sample code.\n", 468 | "\n", 469 | "Context is: # Integrate - Spark\n", 470 | "\n", 471 | "FLAML has integrated Spark for distributed training. There are two main aspects of integration with Spark:\n", 472 | "\n", 473 | "- Use Spark ML estimators for AutoML.\n", 474 | "- Use Spark to run training in parallel spark jobs.\n", 475 | "\n", 476 | "## Spark ML Estimators\n", 477 | "\n", 478 | "FLAML integrates estimators based on Spark ML models. These models are trained in parallel using Spark, so we called them Spark estimators. To use these models, you first need to organize your data in the required format.\n", 479 | "\n", 480 | "### Data\n", 481 | "\n", 482 | "For Spark estimators, AutoML only consumes Spark data. FLAML provides a convenient function `to_pandas_on_spark` in the `flaml.automl.spark.utils` module to convert your data into a pandas-on-spark (`pyspark.pandas`) dataframe/series, which Spark estimators require.\n", 483 | "\n", 484 | "This utility function takes data in the form of a `pandas.Dataframe` or `pyspark.sql.Dataframe` and converts it into a pandas-on-spark dataframe. It also takes `pandas.Series` or `pyspark.sql.Dataframe` and converts it into a [pandas-on-spark](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html) series. If you pass in a `pyspark.pandas.Dataframe`, it will not make any changes.\n", 485 | "\n", 486 | "This function also accepts optional arguments `index_col` and `default_index_type`.\n", 487 | "\n", 488 | "- `index_col` is the column name to use as the index, default is None.\n", 489 | "- `default_index_type` is the default index type, default is \"distributed-sequence\". More info about default index type could be found on Spark official [documentation](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/options.html#default-index-type)\n", 490 | "\n", 491 | "Here is an example code snippet for Spark Data:\n", 492 | "\n", 493 | "```python\n", 494 | "import pandas as pd\n", 495 | "from flaml.automl.spark.utils import to_pandas_on_spark\n", 496 | "\n", 497 | "# Creating a dictionary\n", 498 | "data = {\n", 499 | " \"Square_Feet\": [800, 1200, 1800, 1500, 850],\n", 500 | " \"Age_Years\": [20, 15, 10, 7, 25],\n", 501 | " \"Price\": [100000, 200000, 300000, 240000, 120000],\n", 502 | "}\n", 503 | "\n", 504 | "# Creating a pandas DataFrame\n", 505 | "dataframe = pd.DataFrame(data)\n", 506 | "label = \"Price\"\n", 507 | "\n", 508 | "# Convert to pandas-on-spark dataframe\n", 509 | "psdf = to_pandas_on_spark(dataframe)\n", 510 | "```\n", 511 | "\n", 512 | "To use Spark ML models you need to format your data appropriately. Specifically, use [`VectorAssembler`](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.feature.VectorAssembler.html) to merge all feature columns into a single vector column.\n", 513 | "\n", 514 | "Here is an example of how to use it:\n", 515 | "\n", 516 | "```python\n", 517 | "from pyspark.ml.feature import VectorAssembler\n", 518 | "\n", 519 | "columns = psdf.columns\n", 520 | "feature_cols = [col for col in columns if col != label]\n", 521 | "featurizer = VectorAssembler(inputCols=feature_cols, outputCol=\"features\")\n", 522 | "psdf = featurizer.transform(psdf.to_spark(index_col=\"index\"))[\"index\", \"features\"]\n", 523 | "```\n", 524 | "\n", 525 | "Later in conducting the experiment, use your pandas-on-spark data like non-spark data and pass them using `X_train, y_train` or `dataframe, label`.\n", 526 | "\n", 527 | "### Estimators\n", 528 | "\n", 529 | "#### Model List\n", 530 | "\n", 531 | "- `lgbm_spark`: The class for fine-tuning Spark version LightGBM models, using [SynapseML](https://microsoft.github.io/SynapseML/docs/features/lightgbm/about/) API.\n", 532 | "\n", 533 | "#### Usage\n", 534 | "\n", 535 | "First, prepare your data in the required format as described in the previous section.\n", 536 | "\n", 537 | "By including the models you intend to try in the `estimators_list` argument to `flaml.automl`, FLAML will start trying configurations for these models. If your input is Spark data, FLAML will also use estimators with the `_spark` postfix by default, even if you haven't specified them.\n", 538 | "\n", 539 | "Here is an example code snippet using SparkML models in AutoML:\n", 540 | "\n", 541 | "```python\n", 542 | "import flaml\n", 543 | "\n", 544 | "# prepare your data in pandas-on-spark format as we previously mentioned\n", 545 | "\n", 546 | "automl = flaml.AutoML()\n", 547 | "settings = {\n", 548 | " \"time_budget\": 30,\n", 549 | " \"metric\": \"r2\",\n", 550 | " \"estimator_list\": [\"lgbm_spark\"], # this setting is optional\n", 551 | " \"task\": \"regression\",\n", 552 | "}\n", 553 | "\n", 554 | "automl.fit(\n", 555 | " dataframe=psdf,\n", 556 | " label=label,\n", 557 | " **settings,\n", 558 | ")\n", 559 | "```\n", 560 | "\n", 561 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb)\n", 562 | "\n", 563 | "## Parallel Spark Jobs\n", 564 | "\n", 565 | "You can activate Spark as the parallel backend during parallel tuning in both [AutoML](/docs/Use-Cases/Task-Oriented-AutoML#parallel-tuning) and [Hyperparameter Tuning](/docs/Use-Cases/Tune-User-Defined-Function#parallel-tuning), by setting the `use_spark` to `true`. FLAML will dispatch your job to the distributed Spark backend using [`joblib-spark`](https://github.com/joblib/joblib-spark).\n", 566 | "\n", 567 | "Please note that you should not set `use_spark` to `true` when applying AutoML and Tuning for Spark Data. This is because only SparkML models will be used for Spark Data in AutoML and Tuning. As SparkML models run in parallel, there is no need to distribute them with `use_spark` again.\n", 568 | "\n", 569 | "All the Spark-related arguments are stated below. These arguments are available in both Hyperparameter Tuning and AutoML:\n", 570 | "\n", 571 | "- `use_spark`: boolean, default=False | Whether to use spark to run the training in parallel spark jobs. This can be used to accelerate training on large models and large datasets, but will incur more overhead in time and thus slow down training in some cases. GPU training is not supported yet when use_spark is True. For Spark clusters, by default, we will launch one trial per executor. However, sometimes we want to launch more trials than the number of executors (e.g., local mode). In this case, we can set the environment variable `FLAML_MAX_CONCURRENT` to override the detected `num_executors`. The final number of concurrent trials will be the minimum of `n_concurrent_trials` and `num_executors`.\n", 572 | "- `n_concurrent_trials`: int, default=1 | The number of concurrent trials. When n_concurrent_trials > 1, FLAML performes parallel tuning.\n", 573 | "- `force_cancel`: boolean, default=False | Whether to forcely cancel Spark jobs if the search time exceeded the time budget. Spark jobs include parallel tuning jobs and Spark-based model training jobs.\n", 574 | "\n", 575 | "An example code snippet for using parallel Spark jobs:\n", 576 | "\n", 577 | "```python\n", 578 | "import flaml\n", 579 | "\n", 580 | "automl_experiment = flaml.AutoML()\n", 581 | "automl_settings = {\n", 582 | " \"time_budget\": 30,\n", 583 | " \"metric\": \"r2\",\n", 584 | " \"task\": \"regression\",\n", 585 | " \"n_concurrent_trials\": 2,\n", 586 | " \"use_spark\": True,\n", 587 | " \"force_cancel\": True, # Activating the force_cancel option can immediately halt Spark jobs once they exceed the allocated time_budget.\n", 588 | "}\n", 589 | "\n", 590 | "automl.fit(\n", 591 | " dataframe=dataframe,\n", 592 | " label=label,\n", 593 | " **automl_settings,\n", 594 | ")\n", 595 | "```\n", 596 | "\n", 597 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb)\n", 598 | "\n", 599 | "\n", 600 | "\n", 601 | "--------------------------------------------------------------------------------\n", 602 | "\u001b[33mBoss_Assistant\u001b[0m (to chat_manager):\n", 603 | "\n", 604 | "You're a retrieve augmented coding assistant. You answer user's questions based on your own knowledge and the\n", 605 | "context provided by the user.\n", 606 | "If you can't answer the question with or without the current context, you should reply exactly `UPDATE CONTEXT`.\n", 607 | "For code generation, you must obey the following rules:\n", 608 | "Rule 1. You MUST NOT install any packages because all the packages needed are already installed.\n", 609 | "Rule 2. You must follow the formats below to write your code:\n", 610 | "```language\n", 611 | "# your code\n", 612 | "```\n", 613 | "\n", 614 | "User's question is: How to use spark for parallel training in FLAML? Give me sample code.\n", 615 | "\n", 616 | "Context is: # Integrate - Spark\n", 617 | "\n", 618 | "FLAML has integrated Spark for distributed training. There are two main aspects of integration with Spark:\n", 619 | "\n", 620 | "- Use Spark ML estimators for AutoML.\n", 621 | "- Use Spark to run training in parallel spark jobs.\n", 622 | "\n", 623 | "## Spark ML Estimators\n", 624 | "\n", 625 | "FLAML integrates estimators based on Spark ML models. These models are trained in parallel using Spark, so we called them Spark estimators. To use these models, you first need to organize your data in the required format.\n", 626 | "\n", 627 | "### Data\n", 628 | "\n", 629 | "For Spark estimators, AutoML only consumes Spark data. FLAML provides a convenient function `to_pandas_on_spark` in the `flaml.automl.spark.utils` module to convert your data into a pandas-on-spark (`pyspark.pandas`) dataframe/series, which Spark estimators require.\n", 630 | "\n", 631 | "This utility function takes data in the form of a `pandas.Dataframe` or `pyspark.sql.Dataframe` and converts it into a pandas-on-spark dataframe. It also takes `pandas.Series` or `pyspark.sql.Dataframe` and converts it into a [pandas-on-spark](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html) series. If you pass in a `pyspark.pandas.Dataframe`, it will not make any changes.\n", 632 | "\n", 633 | "This function also accepts optional arguments `index_col` and `default_index_type`.\n", 634 | "\n", 635 | "- `index_col` is the column name to use as the index, default is None.\n", 636 | "- `default_index_type` is the default index type, default is \"distributed-sequence\". More info about default index type could be found on Spark official [documentation](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/options.html#default-index-type)\n", 637 | "\n", 638 | "Here is an example code snippet for Spark Data:\n", 639 | "\n", 640 | "```python\n", 641 | "import pandas as pd\n", 642 | "from flaml.automl.spark.utils import to_pandas_on_spark\n", 643 | "\n", 644 | "# Creating a dictionary\n", 645 | "data = {\n", 646 | " \"Square_Feet\": [800, 1200, 1800, 1500, 850],\n", 647 | " \"Age_Years\": [20, 15, 10, 7, 25],\n", 648 | " \"Price\": [100000, 200000, 300000, 240000, 120000],\n", 649 | "}\n", 650 | "\n", 651 | "# Creating a pandas DataFrame\n", 652 | "dataframe = pd.DataFrame(data)\n", 653 | "label = \"Price\"\n", 654 | "\n", 655 | "# Convert to pandas-on-spark dataframe\n", 656 | "psdf = to_pandas_on_spark(dataframe)\n", 657 | "```\n", 658 | "\n", 659 | "To use Spark ML models you need to format your data appropriately. Specifically, use [`VectorAssembler`](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.feature.VectorAssembler.html) to merge all feature columns into a single vector column.\n", 660 | "\n", 661 | "Here is an example of how to use it:\n", 662 | "\n", 663 | "```python\n", 664 | "from pyspark.ml.feature import VectorAssembler\n", 665 | "\n", 666 | "columns = psdf.columns\n", 667 | "feature_cols = [col for col in columns if col != label]\n", 668 | "featurizer = VectorAssembler(inputCols=feature_cols, outputCol=\"features\")\n", 669 | "psdf = featurizer.transform(psdf.to_spark(index_col=\"index\"))[\"index\", \"features\"]\n", 670 | "```\n", 671 | "\n", 672 | "Later in conducting the experiment, use your pandas-on-spark data like non-spark data and pass them using `X_train, y_train` or `dataframe, label`.\n", 673 | "\n", 674 | "### Estimators\n", 675 | "\n", 676 | "#### Model List\n", 677 | "\n", 678 | "- `lgbm_spark`: The class for fine-tuning Spark version LightGBM models, using [SynapseML](https://microsoft.github.io/SynapseML/docs/features/lightgbm/about/) API.\n", 679 | "\n", 680 | "#### Usage\n", 681 | "\n", 682 | "First, prepare your data in the required format as described in the previous section.\n", 683 | "\n", 684 | "By including the models you intend to try in the `estimators_list` argument to `flaml.automl`, FLAML will start trying configurations for these models. If your input is Spark data, FLAML will also use estimators with the `_spark` postfix by default, even if you haven't specified them.\n", 685 | "\n", 686 | "Here is an example code snippet using SparkML models in AutoML:\n", 687 | "\n", 688 | "```python\n", 689 | "import flaml\n", 690 | "\n", 691 | "# prepare your data in pandas-on-spark format as we previously mentioned\n", 692 | "\n", 693 | "automl = flaml.AutoML()\n", 694 | "settings = {\n", 695 | " \"time_budget\": 30,\n", 696 | " \"metric\": \"r2\",\n", 697 | " \"estimator_list\": [\"lgbm_spark\"], # this setting is optional\n", 698 | " \"task\": \"regression\",\n", 699 | "}\n", 700 | "\n", 701 | "automl.fit(\n", 702 | " dataframe=psdf,\n", 703 | " label=label,\n", 704 | " **settings,\n", 705 | ")\n", 706 | "```\n", 707 | "\n", 708 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb)\n", 709 | "\n", 710 | "## Parallel Spark Jobs\n", 711 | "\n", 712 | "You can activate Spark as the parallel backend during parallel tuning in both [AutoML](/docs/Use-Cases/Task-Oriented-AutoML#parallel-tuning) and [Hyperparameter Tuning](/docs/Use-Cases/Tune-User-Defined-Function#parallel-tuning), by setting the `use_spark` to `true`. FLAML will dispatch your job to the distributed Spark backend using [`joblib-spark`](https://github.com/joblib/joblib-spark).\n", 713 | "\n", 714 | "Please note that you should not set `use_spark` to `true` when applying AutoML and Tuning for Spark Data. This is because only SparkML models will be used for Spark Data in AutoML and Tuning. As SparkML models run in parallel, there is no need to distribute them with `use_spark` again.\n", 715 | "\n", 716 | "All the Spark-related arguments are stated below. These arguments are available in both Hyperparameter Tuning and AutoML:\n", 717 | "\n", 718 | "- `use_spark`: boolean, default=False | Whether to use spark to run the training in parallel spark jobs. This can be used to accelerate training on large models and large datasets, but will incur more overhead in time and thus slow down training in some cases. GPU training is not supported yet when use_spark is True. For Spark clusters, by default, we will launch one trial per executor. However, sometimes we want to launch more trials than the number of executors (e.g., local mode). In this case, we can set the environment variable `FLAML_MAX_CONCURRENT` to override the detected `num_executors`. The final number of concurrent trials will be the minimum of `n_concurrent_trials` and `num_executors`.\n", 719 | "- `n_concurrent_trials`: int, default=1 | The number of concurrent trials. When n_concurrent_trials > 1, FLAML performes parallel tuning.\n", 720 | "- `force_cancel`: boolean, default=False | Whether to forcely cancel Spark jobs if the search time exceeded the time budget. Spark jobs include parallel tuning jobs and Spark-based model training jobs.\n", 721 | "\n", 722 | "An example code snippet for using parallel Spark jobs:\n", 723 | "\n", 724 | "```python\n", 725 | "import flaml\n", 726 | "\n", 727 | "automl_experiment = flaml.AutoML()\n", 728 | "automl_settings = {\n", 729 | " \"time_budget\": 30,\n", 730 | " \"metric\": \"r2\",\n", 731 | " \"task\": \"regression\",\n", 732 | " \"n_concurrent_trials\": 2,\n", 733 | " \"use_spark\": True,\n", 734 | " \"force_cancel\": True, # Activating the force_cancel option can immediately halt Spark jobs once they exceed the allocated time_budget.\n", 735 | "}\n", 736 | "\n", 737 | "automl.fit(\n", 738 | " dataframe=dataframe,\n", 739 | " label=label,\n", 740 | " **automl_settings,\n", 741 | ")\n", 742 | "```\n", 743 | "\n", 744 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb)\n", 745 | "\n", 746 | "\n", 747 | "\n", 748 | "--------------------------------------------------------------------------------\n", 749 | "\u001b[33mProduct_Manager\u001b[0m (to chat_manager):\n", 750 | "\n", 751 | "```python\n", 752 | "from flaml.automl import AutoML\n", 753 | "from flaml.automl.spark.utils import to_pandas_on_spark\n", 754 | "from pyspark.ml.feature import VectorAssembler\n", 755 | "import pandas as pd\n", 756 | "\n", 757 | "# Sample data in a dictionary\n", 758 | "data = {\n", 759 | " \"Square_Feet\": [800, 1200, 1800, 1500, 850],\n", 760 | " \"Age_Years\": [20, 15, 10, 7, 25],\n", 761 | " \"Price\": [100000, 200000, 300000, 240000, 120000],\n", 762 | "}\n", 763 | "\n", 764 | "# Convert dictionary to pandas DataFrame\n", 765 | "dataframe = pd.DataFrame(data)\n", 766 | "label = \"Price\"\n", 767 | "\n", 768 | "# Convert pandas DataFrame to pandas-on-spark DataFrame\n", 769 | "psdf = to_pandas_on_spark(dataframe)\n", 770 | "\n", 771 | "# Use VectorAssembler to merge feature columns into a single vector column\n", 772 | "feature_cols = [col for col in psdf.columns if col != label]\n", 773 | "featurizer = VectorAssembler(inputCols=feature_cols, outputCol=\"features\")\n", 774 | "psdf = featurizer.transform(psdf.to_spark(index_col=\"index\"))[\"index\", \"features\", label]\n", 775 | "\n", 776 | "# Initialize AutoML instance\n", 777 | "automl = AutoML()\n", 778 | "\n", 779 | "# AutoML settings\n", 780 | "automl_settings = {\n", 781 | " \"time_budget\": 30, # Total running time in seconds\n", 782 | " \"metric\": \"r2\", # Evaluation metric\n", 783 | " \"task\": \"regression\",\n", 784 | " \"n_concurrent_trials\": 2, # Number of concurrent Spark jobs\n", 785 | " \"use_spark\": True, # Enable Spark for parallel training\n", 786 | " \"force_cancel\": True, # Force cancel Spark jobs if they exceed the time budget\n", 787 | " \"estimator_list\": [\"lgbm_spark\"] # Optional: Specific estimator to use\n", 788 | "}\n", 789 | "\n", 790 | "# Run AutoML fit with pandas-on-spark dataframe\n", 791 | "automl.fit(\n", 792 | " dataframe=psdf,\n", 793 | " label=label,\n", 794 | " **automl_settings,\n", 795 | ")\n", 796 | "```\n", 797 | "TERMINATE\n", 798 | "\n", 799 | "--------------------------------------------------------------------------------\n" 800 | ] 801 | } 802 | ], 803 | "source": [ 804 | "rag_chat()\n", 805 | "# type exit to terminate the chat" 806 | ] 807 | }, 808 | { 809 | "attachments": {}, 810 | "cell_type": "markdown", 811 | "metadata": {}, 812 | "source": [ 813 | "### Call RetrieveUserProxyAgent while init chat with another user proxy agent\n", 814 | "Sometimes, there might be a need to use RetrieveUserProxyAgent in group chat without initializing the chat with it. In such scenarios, it becomes essential to create a function that wraps the RAG agents and allows them to be called from other agents." 815 | ] 816 | }, 817 | { 818 | "cell_type": "code", 819 | "execution_count": 5, 820 | "metadata": {}, 821 | "outputs": [ 822 | { 823 | "name": "stdout", 824 | "output_type": "stream", 825 | "text": [ 826 | "\u001b[33mBoss\u001b[0m (to chat_manager):\n", 827 | "\n", 828 | "How to use spark for parallel training in FLAML? Give me sample code.\n", 829 | "\n", 830 | "--------------------------------------------------------------------------------\n", 831 | "\u001b[33mProduct_Manager\u001b[0m (to chat_manager):\n", 832 | "\n", 833 | "\u001b[32m***** Suggested function call: retrieve_content *****\u001b[0m\n", 834 | "Arguments: \n", 835 | "{\"message\":\"using Apache Spark for parallel training in FLAML with sample code\"}\n", 836 | "\u001b[32m*****************************************************\u001b[0m\n", 837 | "\n", 838 | "--------------------------------------------------------------------------------\n", 839 | "\u001b[35m\n", 840 | ">>>>>>>> EXECUTING FUNCTION retrieve_content...\u001b[0m\n" 841 | ] 842 | }, 843 | { 844 | "name": "stderr", 845 | "output_type": "stream", 846 | "text": [ 847 | "Number of requested results 3 is greater than number of elements in index 1, updating n_results = 1\n", 848 | "Model gpt4-1106-preview not found. Using cl100k_base encoding.\n" 849 | ] 850 | }, 851 | { 852 | "name": "stdout", 853 | "output_type": "stream", 854 | "text": [ 855 | "VectorDB returns doc_ids: [['bdfbc921']]\n", 856 | "\u001b[32mAdding content of doc bdfbc921 to context.\u001b[0m\n", 857 | "\u001b[33mBoss\u001b[0m (to chat_manager):\n", 858 | "\n", 859 | "\u001b[32m***** Response from calling function (retrieve_content) *****\u001b[0m\n", 860 | "You're a retrieve augmented coding assistant. You answer user's questions based on your own knowledge and the\n", 861 | "context provided by the user.\n", 862 | "If you can't answer the question with or without the current context, you should reply exactly `UPDATE CONTEXT`.\n", 863 | "For code generation, you must obey the following rules:\n", 864 | "Rule 1. You MUST NOT install any packages because all the packages needed are already installed.\n", 865 | "Rule 2. You must follow the formats below to write your code:\n", 866 | "```language\n", 867 | "# your code\n", 868 | "```\n", 869 | "\n", 870 | "User's question is: using Apache Spark for parallel training in FLAML with sample code\n", 871 | "\n", 872 | "Context is: # Integrate - Spark\n", 873 | "\n", 874 | "FLAML has integrated Spark for distributed training. There are two main aspects of integration with Spark:\n", 875 | "\n", 876 | "- Use Spark ML estimators for AutoML.\n", 877 | "- Use Spark to run training in parallel spark jobs.\n", 878 | "\n", 879 | "## Spark ML Estimators\n", 880 | "\n", 881 | "FLAML integrates estimators based on Spark ML models. These models are trained in parallel using Spark, so we called them Spark estimators. To use these models, you first need to organize your data in the required format.\n", 882 | "\n", 883 | "### Data\n", 884 | "\n", 885 | "For Spark estimators, AutoML only consumes Spark data. FLAML provides a convenient function `to_pandas_on_spark` in the `flaml.automl.spark.utils` module to convert your data into a pandas-on-spark (`pyspark.pandas`) dataframe/series, which Spark estimators require.\n", 886 | "\n", 887 | "This utility function takes data in the form of a `pandas.Dataframe` or `pyspark.sql.Dataframe` and converts it into a pandas-on-spark dataframe. It also takes `pandas.Series` or `pyspark.sql.Dataframe` and converts it into a [pandas-on-spark](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html) series. If you pass in a `pyspark.pandas.Dataframe`, it will not make any changes.\n", 888 | "\n", 889 | "This function also accepts optional arguments `index_col` and `default_index_type`.\n", 890 | "\n", 891 | "- `index_col` is the column name to use as the index, default is None.\n", 892 | "- `default_index_type` is the default index type, default is \"distributed-sequence\". More info about default index type could be found on Spark official [documentation](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/options.html#default-index-type)\n", 893 | "\n", 894 | "Here is an example code snippet for Spark Data:\n", 895 | "\n", 896 | "```python\n", 897 | "import pandas as pd\n", 898 | "from flaml.automl.spark.utils import to_pandas_on_spark\n", 899 | "\n", 900 | "# Creating a dictionary\n", 901 | "data = {\n", 902 | " \"Square_Feet\": [800, 1200, 1800, 1500, 850],\n", 903 | " \"Age_Years\": [20, 15, 10, 7, 25],\n", 904 | " \"Price\": [100000, 200000, 300000, 240000, 120000],\n", 905 | "}\n", 906 | "\n", 907 | "# Creating a pandas DataFrame\n", 908 | "dataframe = pd.DataFrame(data)\n", 909 | "label = \"Price\"\n", 910 | "\n", 911 | "# Convert to pandas-on-spark dataframe\n", 912 | "psdf = to_pandas_on_spark(dataframe)\n", 913 | "```\n", 914 | "\n", 915 | "To use Spark ML models you need to format your data appropriately. Specifically, use [`VectorAssembler`](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.feature.VectorAssembler.html) to merge all feature columns into a single vector column.\n", 916 | "\n", 917 | "Here is an example of how to use it:\n", 918 | "\n", 919 | "```python\n", 920 | "from pyspark.ml.feature import VectorAssembler\n", 921 | "\n", 922 | "columns = psdf.columns\n", 923 | "feature_cols = [col for col in columns if col != label]\n", 924 | "featurizer = VectorAssembler(inputCols=feature_cols, outputCol=\"features\")\n", 925 | "psdf = featurizer.transform(psdf.to_spark(index_col=\"index\"))[\"index\", \"features\"]\n", 926 | "```\n", 927 | "\n", 928 | "Later in conducting the experiment, use your pandas-on-spark data like non-spark data and pass them using `X_train, y_train` or `dataframe, label`.\n", 929 | "\n", 930 | "### Estimators\n", 931 | "\n", 932 | "#### Model List\n", 933 | "\n", 934 | "- `lgbm_spark`: The class for fine-tuning Spark version LightGBM models, using [SynapseML](https://microsoft.github.io/SynapseML/docs/features/lightgbm/about/) API.\n", 935 | "\n", 936 | "#### Usage\n", 937 | "\n", 938 | "First, prepare your data in the required format as described in the previous section.\n", 939 | "\n", 940 | "By including the models you intend to try in the `estimators_list` argument to `flaml.automl`, FLAML will start trying configurations for these models. If your input is Spark data, FLAML will also use estimators with the `_spark` postfix by default, even if you haven't specified them.\n", 941 | "\n", 942 | "Here is an example code snippet using SparkML models in AutoML:\n", 943 | "\n", 944 | "```python\n", 945 | "import flaml\n", 946 | "\n", 947 | "# prepare your data in pandas-on-spark format as we previously mentioned\n", 948 | "\n", 949 | "automl = flaml.AutoML()\n", 950 | "settings = {\n", 951 | " \"time_budget\": 30,\n", 952 | " \"metric\": \"r2\",\n", 953 | " \"estimator_list\": [\"lgbm_spark\"], # this setting is optional\n", 954 | " \"task\": \"regression\",\n", 955 | "}\n", 956 | "\n", 957 | "automl.fit(\n", 958 | " dataframe=psdf,\n", 959 | " label=label,\n", 960 | " **settings,\n", 961 | ")\n", 962 | "```\n", 963 | "\n", 964 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb)\n", 965 | "\n", 966 | "## Parallel Spark Jobs\n", 967 | "\n", 968 | "You can activate Spark as the parallel backend during parallel tuning in both [AutoML](/docs/Use-Cases/Task-Oriented-AutoML#parallel-tuning) and [Hyperparameter Tuning](/docs/Use-Cases/Tune-User-Defined-Function#parallel-tuning), by setting the `use_spark` to `true`. FLAML will dispatch your job to the distributed Spark backend using [`joblib-spark`](https://github.com/joblib/joblib-spark).\n", 969 | "\n", 970 | "Please note that you should not set `use_spark` to `true` when applying AutoML and Tuning for Spark Data. This is because only SparkML models will be used for Spark Data in AutoML and Tuning. As SparkML models run in parallel, there is no need to distribute them with `use_spark` again.\n", 971 | "\n", 972 | "All the Spark-related arguments are stated below. These arguments are available in both Hyperparameter Tuning and AutoML:\n", 973 | "\n", 974 | "- `use_spark`: boolean, default=False | Whether to use spark to run the training in parallel spark jobs. This can be used to accelerate training on large models and large datasets, but will incur more overhead in time and thus slow down training in some cases. GPU training is not supported yet when use_spark is True. For Spark clusters, by default, we will launch one trial per executor. However, sometimes we want to launch more trials than the number of executors (e.g., local mode). In this case, we can set the environment variable `FLAML_MAX_CONCURRENT` to override the detected `num_executors`. The final number of concurrent trials will be the minimum of `n_concurrent_trials` and `num_executors`.\n", 975 | "- `n_concurrent_trials`: int, default=1 | The number of concurrent trials. When n_concurrent_trials > 1, FLAML performes parallel tuning.\n", 976 | "- `force_cancel`: boolean, default=False | Whether to forcely cancel Spark jobs if the search time exceeded the time budget. Spark jobs include parallel tuning jobs and Spark-based model training jobs.\n", 977 | "\n", 978 | "An example code snippet for using parallel Spark jobs:\n", 979 | "\n", 980 | "```python\n", 981 | "import flaml\n", 982 | "\n", 983 | "automl_experiment = flaml.AutoML()\n", 984 | "automl_settings = {\n", 985 | " \"time_budget\": 30,\n", 986 | " \"metric\": \"r2\",\n", 987 | " \"task\": \"regression\",\n", 988 | " \"n_concurrent_trials\": 2,\n", 989 | " \"use_spark\": True,\n", 990 | " \"force_cancel\": True, # Activating the force_cancel option can immediately halt Spark jobs once they exceed the allocated time_budget.\n", 991 | "}\n", 992 | "\n", 993 | "automl.fit(\n", 994 | " dataframe=dataframe,\n", 995 | " label=label,\n", 996 | " **automl_settings,\n", 997 | ")\n", 998 | "```\n", 999 | "\n", 1000 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb)\n", 1001 | "\n", 1002 | "\n", 1003 | "\u001b[32m*************************************************************\u001b[0m\n", 1004 | "\n", 1005 | "--------------------------------------------------------------------------------\n", 1006 | "\u001b[33mBoss\u001b[0m (to chat_manager):\n", 1007 | "\n", 1008 | "\u001b[32m***** Response from calling function (retrieve_content) *****\u001b[0m\n", 1009 | "You're a retrieve augmented coding assistant. You answer user's questions based on your own knowledge and the\n", 1010 | "context provided by the user.\n", 1011 | "If you can't answer the question with or without the current context, you should reply exactly `UPDATE CONTEXT`.\n", 1012 | "For code generation, you must obey the following rules:\n", 1013 | "Rule 1. You MUST NOT install any packages because all the packages needed are already installed.\n", 1014 | "Rule 2. You must follow the formats below to write your code:\n", 1015 | "```language\n", 1016 | "# your code\n", 1017 | "```\n", 1018 | "\n", 1019 | "User's question is: using Apache Spark for parallel training in FLAML with sample code\n", 1020 | "\n", 1021 | "Context is: # Integrate - Spark\n", 1022 | "\n", 1023 | "FLAML has integrated Spark for distributed training. There are two main aspects of integration with Spark:\n", 1024 | "\n", 1025 | "- Use Spark ML estimators for AutoML.\n", 1026 | "- Use Spark to run training in parallel spark jobs.\n", 1027 | "\n", 1028 | "## Spark ML Estimators\n", 1029 | "\n", 1030 | "FLAML integrates estimators based on Spark ML models. These models are trained in parallel using Spark, so we called them Spark estimators. To use these models, you first need to organize your data in the required format.\n", 1031 | "\n", 1032 | "### Data\n", 1033 | "\n", 1034 | "For Spark estimators, AutoML only consumes Spark data. FLAML provides a convenient function `to_pandas_on_spark` in the `flaml.automl.spark.utils` module to convert your data into a pandas-on-spark (`pyspark.pandas`) dataframe/series, which Spark estimators require.\n", 1035 | "\n", 1036 | "This utility function takes data in the form of a `pandas.Dataframe` or `pyspark.sql.Dataframe` and converts it into a pandas-on-spark dataframe. It also takes `pandas.Series` or `pyspark.sql.Dataframe` and converts it into a [pandas-on-spark](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html) series. If you pass in a `pyspark.pandas.Dataframe`, it will not make any changes.\n", 1037 | "\n", 1038 | "This function also accepts optional arguments `index_col` and `default_index_type`.\n", 1039 | "\n", 1040 | "- `index_col` is the column name to use as the index, default is None.\n", 1041 | "- `default_index_type` is the default index type, default is \"distributed-sequence\". More info about default index type could be found on Spark official [documentation](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/options.html#default-index-type)\n", 1042 | "\n", 1043 | "Here is an example code snippet for Spark Data:\n", 1044 | "\n", 1045 | "```python\n", 1046 | "import pandas as pd\n", 1047 | "from flaml.automl.spark.utils import to_pandas_on_spark\n", 1048 | "\n", 1049 | "# Creating a dictionary\n", 1050 | "data = {\n", 1051 | " \"Square_Feet\": [800, 1200, 1800, 1500, 850],\n", 1052 | " \"Age_Years\": [20, 15, 10, 7, 25],\n", 1053 | " \"Price\": [100000, 200000, 300000, 240000, 120000],\n", 1054 | "}\n", 1055 | "\n", 1056 | "# Creating a pandas DataFrame\n", 1057 | "dataframe = pd.DataFrame(data)\n", 1058 | "label = \"Price\"\n", 1059 | "\n", 1060 | "# Convert to pandas-on-spark dataframe\n", 1061 | "psdf = to_pandas_on_spark(dataframe)\n", 1062 | "```\n", 1063 | "\n", 1064 | "To use Spark ML models you need to format your data appropriately. Specifically, use [`VectorAssembler`](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.feature.VectorAssembler.html) to merge all feature columns into a single vector column.\n", 1065 | "\n", 1066 | "Here is an example of how to use it:\n", 1067 | "\n", 1068 | "```python\n", 1069 | "from pyspark.ml.feature import VectorAssembler\n", 1070 | "\n", 1071 | "columns = psdf.columns\n", 1072 | "feature_cols = [col for col in columns if col != label]\n", 1073 | "featurizer = VectorAssembler(inputCols=feature_cols, outputCol=\"features\")\n", 1074 | "psdf = featurizer.transform(psdf.to_spark(index_col=\"index\"))[\"index\", \"features\"]\n", 1075 | "```\n", 1076 | "\n", 1077 | "Later in conducting the experiment, use your pandas-on-spark data like non-spark data and pass them using `X_train, y_train` or `dataframe, label`.\n", 1078 | "\n", 1079 | "### Estimators\n", 1080 | "\n", 1081 | "#### Model List\n", 1082 | "\n", 1083 | "- `lgbm_spark`: The class for fine-tuning Spark version LightGBM models, using [SynapseML](https://microsoft.github.io/SynapseML/docs/features/lightgbm/about/) API.\n", 1084 | "\n", 1085 | "#### Usage\n", 1086 | "\n", 1087 | "First, prepare your data in the required format as described in the previous section.\n", 1088 | "\n", 1089 | "By including the models you intend to try in the `estimators_list` argument to `flaml.automl`, FLAML will start trying configurations for these models. If your input is Spark data, FLAML will also use estimators with the `_spark` postfix by default, even if you haven't specified them.\n", 1090 | "\n", 1091 | "Here is an example code snippet using SparkML models in AutoML:\n", 1092 | "\n", 1093 | "```python\n", 1094 | "import flaml\n", 1095 | "\n", 1096 | "# prepare your data in pandas-on-spark format as we previously mentioned\n", 1097 | "\n", 1098 | "automl = flaml.AutoML()\n", 1099 | "settings = {\n", 1100 | " \"time_budget\": 30,\n", 1101 | " \"metric\": \"r2\",\n", 1102 | " \"estimator_list\": [\"lgbm_spark\"], # this setting is optional\n", 1103 | " \"task\": \"regression\",\n", 1104 | "}\n", 1105 | "\n", 1106 | "automl.fit(\n", 1107 | " dataframe=psdf,\n", 1108 | " label=label,\n", 1109 | " **settings,\n", 1110 | ")\n", 1111 | "```\n", 1112 | "\n", 1113 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/automl_bankrupt_synapseml.ipynb)\n", 1114 | "\n", 1115 | "## Parallel Spark Jobs\n", 1116 | "\n", 1117 | "You can activate Spark as the parallel backend during parallel tuning in both [AutoML](/docs/Use-Cases/Task-Oriented-AutoML#parallel-tuning) and [Hyperparameter Tuning](/docs/Use-Cases/Tune-User-Defined-Function#parallel-tuning), by setting the `use_spark` to `true`. FLAML will dispatch your job to the distributed Spark backend using [`joblib-spark`](https://github.com/joblib/joblib-spark).\n", 1118 | "\n", 1119 | "Please note that you should not set `use_spark` to `true` when applying AutoML and Tuning for Spark Data. This is because only SparkML models will be used for Spark Data in AutoML and Tuning. As SparkML models run in parallel, there is no need to distribute them with `use_spark` again.\n", 1120 | "\n", 1121 | "All the Spark-related arguments are stated below. These arguments are available in both Hyperparameter Tuning and AutoML:\n", 1122 | "\n", 1123 | "- `use_spark`: boolean, default=False | Whether to use spark to run the training in parallel spark jobs. This can be used to accelerate training on large models and large datasets, but will incur more overhead in time and thus slow down training in some cases. GPU training is not supported yet when use_spark is True. For Spark clusters, by default, we will launch one trial per executor. However, sometimes we want to launch more trials than the number of executors (e.g., local mode). In this case, we can set the environment variable `FLAML_MAX_CONCURRENT` to override the detected `num_executors`. The final number of concurrent trials will be the minimum of `n_concurrent_trials` and `num_executors`.\n", 1124 | "- `n_concurrent_trials`: int, default=1 | The number of concurrent trials. When n_concurrent_trials > 1, FLAML performes parallel tuning.\n", 1125 | "- `force_cancel`: boolean, default=False | Whether to forcely cancel Spark jobs if the search time exceeded the time budget. Spark jobs include parallel tuning jobs and Spark-based model training jobs.\n", 1126 | "\n", 1127 | "An example code snippet for using parallel Spark jobs:\n", 1128 | "\n", 1129 | "```python\n", 1130 | "import flaml\n", 1131 | "\n", 1132 | "automl_experiment = flaml.AutoML()\n", 1133 | "automl_settings = {\n", 1134 | " \"time_budget\": 30,\n", 1135 | " \"metric\": \"r2\",\n", 1136 | " \"task\": \"regression\",\n", 1137 | " \"n_concurrent_trials\": 2,\n", 1138 | " \"use_spark\": True,\n", 1139 | " \"force_cancel\": True, # Activating the force_cancel option can immediately halt Spark jobs once they exceed the allocated time_budget.\n", 1140 | "}\n", 1141 | "\n", 1142 | "automl.fit(\n", 1143 | " dataframe=dataframe,\n", 1144 | " label=label,\n", 1145 | " **automl_settings,\n", 1146 | ")\n", 1147 | "```\n", 1148 | "\n", 1149 | "[Link to notebook](https://github.com/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb) | [Open in colab](https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/integrate_spark.ipynb)\n", 1150 | "\n", 1151 | "\n", 1152 | "\u001b[32m*************************************************************\u001b[0m\n", 1153 | "\n", 1154 | "--------------------------------------------------------------------------------\n", 1155 | "\u001b[33mProduct_Manager\u001b[0m (to chat_manager):\n", 1156 | "\n", 1157 | "To use Apache Spark for parallel training in FLAML, you can follow these steps:\n", 1158 | "\n", 1159 | "1. Ensure your data is in the required pandas-on-spark format.\n", 1160 | "2. Use Spark ML estimators by including them in the `estimator_list`.\n", 1161 | "3. Set `use_spark` to `True` for parallel tuning.\n", 1162 | "\n", 1163 | "Here's a sample code demonstrating how to use Spark for parallel training in FLAML:\n", 1164 | "\n", 1165 | "```python\n", 1166 | "import flaml\n", 1167 | "from flaml.automl.spark.utils import to_pandas_on_spark\n", 1168 | "import pandas as pd\n", 1169 | "from pyspark.ml.feature import VectorAssembler\n", 1170 | "\n", 1171 | "# Sample data in a pandas DataFrame\n", 1172 | "data = {\n", 1173 | " \"Square_Feet\": [800, 1200, 1800, 1500, 850],\n", 1174 | " \"Age_Years\": [20, 15, 10, 7, 25],\n", 1175 | " \"Price\": [100000, 200000, 300000, 240000, 120000],\n", 1176 | "}\n", 1177 | "label = \"Price\"\n", 1178 | "\n", 1179 | "# Creating a pandas DataFrame\n", 1180 | "dataframe = pd.DataFrame(data)\n", 1181 | "\n", 1182 | "# Convert to pandas-on-spark dataframe\n", 1183 | "psdf = to_pandas_on_spark(dataframe)\n", 1184 | "\n", 1185 | "# Prepare features using VectorAssembler\n", 1186 | "columns = psdf.columns\n", 1187 | "feature_cols = [col for col in columns if col != label]\n", 1188 | "featurizer = VectorAssembler(inputCols=feature_cols, outputCol=\"features\")\n", 1189 | "psdf = featurizer.transform(psdf.to_spark(index_col=\"index\"))[\"index\", \"features\"]\n", 1190 | "\n", 1191 | "# Initialize AutoML\n", 1192 | "automl = flaml.AutoML()\n", 1193 | "\n", 1194 | "# Configure settings for AutoML\n", 1195 | "settings = {\n", 1196 | " \"time_budget\": 30, # time budget in seconds\n", 1197 | " \"metric\": \"r2\",\n", 1198 | " \"estimator_list\": [\"lgbm_spark\"], # using Spark ML estimators\n", 1199 | " \"task\": \"regression\",\n", 1200 | " \"n_concurrent_trials\": 2, # number of parallel trials\n", 1201 | " \"use_spark\": True, # enable parallel training using Spark\n", 1202 | " \"force_cancel\": True, # force cancel Spark jobs if time_budget is exceeded\n", 1203 | "}\n", 1204 | "\n", 1205 | "# Start the training\n", 1206 | "automl.fit(dataframe=psdf, label=label, **settings)\n", 1207 | "```\n", 1208 | "\n", 1209 | "In this code snippet:\n", 1210 | "- The `to_pandas_on_spark` function is used to convert the pandas DataFrame to a pandas-on-spark DataFrame.\n", 1211 | "- `VectorAssembler` is used to transform feature columns into a single vector column.\n", 1212 | "- The `AutoML` object is created, and settings are configured for the AutoML run, including setting `use_spark` to `True` for parallel training.\n", 1213 | "- The `fit` method is called to start the automated machine learning process.\n", 1214 | "\n", 1215 | "By using these settings, FLAML will train the models in parallel using Spark, which can accelerate the training process on large models and datasets.\n", 1216 | "\n", 1217 | "TERMINATE\n", 1218 | "\n", 1219 | "--------------------------------------------------------------------------------\n" 1220 | ] 1221 | } 1222 | ], 1223 | "source": [ 1224 | "call_rag_chat()" 1225 | ] 1226 | }, 1227 | { 1228 | "cell_type": "code", 1229 | "execution_count": null, 1230 | "metadata": {}, 1231 | "outputs": [], 1232 | "source": [] 1233 | } 1234 | ], 1235 | "metadata": { 1236 | "front_matter": { 1237 | "description": "Implement and manage a multi-agent chat system using AutoGen, where AI assistants retrieve information, generate code, and interact collaboratively to solve complex tasks, especially in areas not covered by their training data.", 1238 | "tags": [ 1239 | "group chat", 1240 | "orchestration", 1241 | "RAG" 1242 | ] 1243 | }, 1244 | "kernelspec": { 1245 | "display_name": "flaml", 1246 | "language": "python", 1247 | "name": "python3" 1248 | }, 1249 | "language_info": { 1250 | "codemirror_mode": { 1251 | "name": "ipython", 1252 | "version": 3 1253 | }, 1254 | "file_extension": ".py", 1255 | "mimetype": "text/x-python", 1256 | "name": "python", 1257 | "nbconvert_exporter": "python", 1258 | "pygments_lexer": "ipython3", 1259 | "version": "3.10.13" 1260 | } 1261 | }, 1262 | "nbformat": 4, 1263 | "nbformat_minor": 2 1264 | } 1265 | --------------------------------------------------------------------------------