├── README.md └── assets ├── app&eval.png ├── architecture-1.png ├── architecture.png ├── capability.png ├── planning.png ├── table.png └── trend.png /README.md: -------------------------------------------------------------------------------- 1 | # A Survey on LLM-based Autonomous Agents 2 | 3 | ![Growth Trend](assets/trend.png) 4 | 5 | Autonomous agents are designed to achieve specific objectives through self-guided instructions. With the emergence and growth of large language models (LLMs), there is a growing trend in utilizing LLMs as fundamental controllers for these autonomous agents. While previous studies in this field have achieved remarkable successes, they remain independent proposals with little effort devoted to a systematic analysis. To bridge this gap, we conduct a comprehensive survey study, focusing on the construction, application, and evaluation of LLM-based autonomous agents. In particular, we first explore the essential components of an AI agent, including a profile module, a memory module, a planning module, and an action module. We further investigate the application of LLM-based autonomous agents in the domains of natural sciences, social sciences, and engineering. Subsequently, we delve into a discussion of the evaluation strategies employed in this field, encompassing both subjective and objective methods. Our survey aims to serve as a resource for researchers and practitioners, providing insights, related references, and continuous updates on this exciting and rapidly evolving field. 6 | 7 | **📍 This is the first released and published survey paper in the field of LLM-based autonomous agents.** 8 | 9 | Paper link: [A Survey on Large Language Model based Autonomous Agents](https://arxiv.org/abs/2308.11432) 10 | 11 | 12 | ## Update Records 13 | - 🔥 [25/3/2024] Our survey paper has been accepted by Frontiers of Computer Science, which is the first published survey paper in the field of LLM-based agents. 14 | 15 | - 🔥 [9/8/2023] The second version of our survey has been released on arXiv. 16 |
17 | Updated contents 18 | 19 | - **📚 Additional References** 20 | - We have added 31 new works until 9/1/2023 to make the survey more comprehensive and up-to-date. 21 | 22 | - **📊 New Figures** 23 | - **Figure 3:** This is a new figure illustrating the differences and similarities between various planning approaches. This helps in gaining a clearer understanding of the comparisons between different planning methods. 24 | ![single-path and multi-path reasoning](assets/planning.png) 25 | - **Figure 4:** This is a new figure that describes the evolutionary path of model capability acquisition from the "Machine Learning era" to the "Large Language Model era" and then to the "Agent era." Specifically, a new concept, "mechanism engineering," has been introduced, which, along with "parameter learning" and "prompt engineering," forms part of this evolutionary path. 26 | ![Capabilities Acquisition](assets/capability.png) 27 | 28 | - **🔍 Optimized Classification System** 29 | - We have slightly modified the classification system in our survey to make it more logical and organized. 30 |
31 | 32 | - 🔥 [8/23/2023] The first version of our survey has been released on arXiv.
33 | 34 | 35 | 36 | ## Table of Content 37 | 38 | 39 | - [🤖 Construction of LLM-based Autonomous Agent](#-construction-of-llm-based-autonomous-agent) 40 | - [📍 Applications of LLM-based Autonomous Agent](#-applications-of-llm-based-autonomous-agent) 41 | - [📊 Evaluation on LLM-based Autonomous Agent](#-evaluation-on-llm-based-autonomous-agent) 42 | - [🌐 More Comprehensive Summarization](#-more-comprehensive-summarization) 43 | - [👨‍👨‍👧‍👦 Maintainers](#-maintainers) 44 | - [📚 Citation](#-citation) 45 | - [💪 How to Contribute](#-how-to-contribute) 46 | - [🫡 Acknowledgement](#-acknowledgement) 47 | - [📧 Contact Us](#-contact-us) 48 | 49 | 50 | 51 | ## 🤖 Construction of LLM-based Autonomous Agent 52 | ![Architecture Design](assets/architecture-1.png) 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 199 | 200 | 201 | 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | 211 | 212 | 213 | 214 | 215 | 216 | 217 | 218 | 219 | 220 | 221 | 222 | 223 | 224 | 225 | 226 | 227 | 228 | 229 | 230 | 231 | 232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 240 | 241 | 242 | 243 | 244 | 245 | 246 | 247 | 248 | 249 | 250 | 251 | 252 | 253 | 254 | 255 | 256 | 257 | 258 | 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 | 275 | 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | 284 | 285 | 286 | 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 | 297 | 298 | 299 | 300 | 301 | 302 | 303 | 304 | 305 | 306 | 307 | 308 | 309 | 310 | 311 | 312 | 313 | 314 | 315 | 316 | 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | 327 | 328 | 329 | 330 | 331 | 332 | 333 | 334 | 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | 343 | 344 | 345 | 346 | 347 | 348 | 349 | 350 | 351 | 352 | 353 | 354 | 355 | 356 | 357 | 358 | 359 | 360 | 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 | 369 | 370 | 371 | 372 | 373 | 374 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | 382 | 383 | 384 | 385 | 386 | 387 | 388 | 389 | 390 | 391 | 392 | 393 | 394 | 395 | 396 | 397 | 398 | 399 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 407 | 408 | 409 | 410 | 411 | 412 | 413 | 414 | 415 | 416 | 417 | 418 | 419 | 420 | 421 | 422 | 423 | 424 | 425 | 426 | 427 | 428 | 429 | 430 | 431 | 432 | 433 | 434 | 435 | 436 | 437 | 438 | 439 | 440 | 441 | 442 | 443 | 444 | 445 | 446 | 447 | 448 | 449 | 450 | 451 | 452 | 453 | 454 | 455 | 456 | 457 | 458 | 459 | 460 | 461 | 462 | 463 | 464 | 465 | 466 | 467 | 468 | 469 | 470 | 471 | 472 | 473 | 474 | 475 | 476 | 477 | 478 | 479 | 480 | 481 | 482 | 483 | 484 | 485 | 486 | 487 | 488 | 489 | 490 | 491 | 492 | 493 | 494 | 495 | 496 | 497 | 498 | 499 | 500 | 501 | 502 | 503 | 504 | 505 | 506 | 507 | 508 | 509 | 510 |
ModelProfileMemoryPlanningActionCAPaperCode
OperationStructure
WebGPT----w/ toolsw/ fine-tuningPaper-
SayCan---w/o feedbackw/o toolsw/o fine-tuningPaperCode
MRKL---w/o feedbackw/ tools-Paper-
Inner Monologue---w/ feedbackw/o toolsw/o fine-tuningPaperCode
Social SimulacraGPT-Generated---w/o tools-Paper-
ReAct---w/ feedbackw/ toolsw/ fine-tuningPaperCode
LLM Planner---w/ feedbackw/o toolsEnvironment feedbackPaperCode
MALLM-Read/WriteHybrid-w/o tools-Paper-
aiflows-Read/Write/
Reflection		Hybridw/ feedbackw/ tools-PaperCode
DEPS---w/ feedbackw/o toolsw/o fine-tuningPaperCode
Toolformer---w/o feedbackw/ toolsw/ fine-tuningPaperCode
Reflexion-Read/Write/
Reflection		Hybridw/ feedbackw/o toolsw/o fine-tuningPaperCode
CAMELHandcrafting & GPT-Generated--w/ feedbackw/o tools-PaperCode
API-Bank---w/ feedbackw/ toolsw/o fine-tuningPaper-
Chameleon---w/o feedbackw/ tools-PaperCode
ViperGPT----w/ tools-PaperCode
HuggingGPT--Unifiedw/o feedbackw/ tools-PaperCode
Generative AgentsHandcraftingRead/Write/
Reflection		Hybridw/ feedbackw/o tools-PaperCode
LLM+P---w/o feedbackw/o tools-Paper-
ChemCrow---w/ feedbackw/ tools-PaperCode
OpenAGI---w/ feedbackw/ toolsw/ fine-tuningPaperCode
AutoGPT-Read/WriteHybridw/ feedbackw/ toolsw/o fine-tuning-Code
SCM-Read/WriteHybrid-w/o tools-PaperCode
Socially Alignment-Read/WriteHybrid-w/o toolsExamplePaperCode
GITM-Read/Write/
Reflection		Hybridw/ feedbackw/o toolsw/ fine-tuningPaperCode
Voyager-Read/Write/
Reflection		Hybridw/ feedbackw/o toolsw/o fine-tuningPaperCode
Introspective Tips---w/ feedbackw/o toolsw/o fine-tuningPaper-
RET-LLM-Read/WriteHybrid-w/o toolsw/ fine-tuningPaper-
ChatDB-Read/WriteHybridw/ feedbackw/ tools-Paper-
S3Dataset alignmentRead/Write/
Reflection		Hybrid-w/o toolsw/ fine-tuningPaper-
ChatDevHandcraftingRead/Write/
Reflection		Hybridw/ feedbackw/o toolsw/o fine-tuningPaperCode
ToolLLM---w/ feedbackw/ toolsw/ fine-tuningPaperCode
MemoryBank-Read/Write/
Reflection		Hybrid-w/o tools-PaperCode
MetaGPTHandcraftingRead/Write/
Reflection		Hybridw/ feedbackw/ tools-PaperCode
L2MACHandcraftingRead/Write/
Reflection		Hybridw/ feedbackw/ tools-PaperCode
LEO---w/ feedbackw/o toolsw/ fine-tuningPaperCode
JARVIS-1-Read/Write/
Reflection		Hybridw/ feedbackw/ toolsw/o fine-tuningPaperCode
CLOVA-Read/Write/
Reflection		Hybridw/ feedbackw/ toolsw/ fine-tuningPaperCode
LearnAct---w/ feedbackw/ toolsw/ fine-tuningPaperCode
AgentSquare-Read/WriteHybridw/ feedbackw/ tools-PaperCode
511 | 512 | * More papers can be found at [More comprehensive Summarization](#-more-comprehensive-summarization). 513 | * CA means the strategy of model capability acquisition. 514 | 515 | ## 📍 Applications of LLM-based Autonomous Agent 516 | 517 | 518 | 519 | 520 | 521 | 522 | 523 | 524 | 525 | 526 | 527 | 528 | 529 | 530 | 531 | 532 | 533 | 534 | 535 | 536 | 537 | 538 | 539 | 540 | 541 | 542 | 543 | 544 | 545 | 546 | 547 | 548 | 549 | 550 | 551 | 552 | 553 | 554 | 555 | 556 | 557 | 558 | 559 | 560 | 561 | 562 | 563 | 564 | 565 | 566 | 567 | 568 | 569 | 570 | 571 | 572 | 573 | 574 | 575 | 576 | 577 | 578 | 579 | 580 | 581 | 582 | 583 | 584 | 585 | 586 | 587 | 588 | 589 | 590 | 591 | 592 | 593 | 594 | 595 | 596 | 597 | 598 | 599 | 600 | 601 | 602 | 603 | 604 | 605 | 606 | 607 | 608 | 609 | 610 | 611 | 612 | 613 | 614 | 615 | 616 | 617 | 618 | 619 | 620 | 621 | 622 | 623 | 624 | 625 | 626 | 627 | 628 | 629 | 630 | 631 | 632 | 633 | 634 | 635 | 636 | 637 | 638 | 639 | 640 | 641 | 642 | 643 | 644 | 645 | 646 | 647 | 648 | 649 | 650 | 651 | 652 | 653 | 654 | 655 | 656 | 657 | 658 | 659 | 660 | 661 | 662 | 663 | 664 | 665 | 666 | 667 | 668 | 669 | 670 | 671 | 672 | 673 | 674 | 675 | 676 | 677 | 678 | 679 | 680 | 681 | 682 | 683 | 684 | 685 | 686 | 687 | 688 | 689 | 690 | 691 | 692 | 693 | 694 | 695 | 696 | 697 | 698 | 699 | 700 | 701 | 702 | 703 | 704 | 705 | 706 | 707 | 708 | 709 | 710 | 711 | 712 | 713 | 714 | 715 | 716 | 717 | 718 | 719 | 720 | 721 | 722 | 723 | 724 | 725 | 726 | 727 | 728 | 729 | 730 | 731 | 732 | 733 | 734 | 735 | 736 | 737 | 738 | 739 | 740 | 741 | 742 | 743 | 744 | 745 | 746 | 747 | 748 | 749 | 750 | 751 | 752 | 753 | 754 | 755 | 756 | 757 | 758 | 759 | 760 | 761 | 762 | 763 | 764 | 765 | 766 | 767 | 768 | 769 | 770 | 771 | 772 | 773 | 774 | 775 | 776 | 777 | 778 | 779 | 780 | 781 | 782 | 783 | 784 | 785 | 786 | 787 | 788 | 789 | 790 | 791 | 792 | 793 | 794 | 795 | 796 | 797 | 798 | 799 | 800 | 801 | 802 | 803 | 804 | 805 | 806 | 807 | 808 | 809 | 810 | 811 | 812 | 813 | 814 | 815 | 816 | 817 | 818 | 819 | 820 | 821 | 822 | 823 | 824 | 825 | 826 | 827 | 828 | 829 | 830 | 831 | 832 | 833 | 834 | 835 | 836 | 837 | 838 | 839 | 840 | 841 | 842 | 843 | 844 | 845 | 846 | 847 | 848 | 849 | 850 | 851 | 852 | 853 | 854 | 855 | 856 | 857 | 858 | 859 | 860 | 861 | 862 | 863 | 864 | 865 | 866 | 867 | 868 | 869 | 870 | 871 | 872 | 873 | 874 | 875 | 876 | 877 | 878 | 879 | 880 | 881 | 882 | 883 | 884 | 885 | 886 | 887 | 888 | 889 | 890 | 891 | 892 | 893 | 894 | 895 | 896 | 897 | 898 | 899 | 900 | 901 | 902 | 903 | 904 | 905 | 906 | 907 | 908 | 909 | 910 | 911 | 912 | 913 | 914 | 915 | 916 | 917 | 918 | 919 | 920 | 921 | 922 | 923 | 924 | 925 | 926 | 927 | 928 | 929 | 930 | 931 | 932 | 933 | 934 | 935 | 936 | 937 | 938 | 939 | 940 | 941 | 942 | 943 | 944 | 945 | 946 | 947 | 948 | 949 | 950 | 951 | 952 | 953 | 954 | 955 | 956 | 957 | 958 | 959 | 960 | 961 | 962 | 963 | 964 | 965 | 966 | 967 | 968 | 969 | 970 | 971 | 972 | 973 | 974 | 975 | 976 | 977 | 978 | 979 | 980 | 981 | 982 | 983 | 984 | 985 | 986 | 987 | 988 | 989 | 990 | 991 | 992 | 993 | 994 | 995 | 996 | 997 | 998 | 999 | 1000 | 1001 | 1002 | 1003 | 1004 | 1005 | 1006 | 1007 | 1008 | 1009 | 1010 | 1011 | 1012 | 1013 | 1014 | 1015 | 1016 | 1017 | 1018 | 1019 | 1020 | 1021 | 1022 | 1023 | 1024 | 1025 | 1026 | 1027 | 1028 | 1029 | 1030 | 1031 | 1032 | 1033 | 1034 | 1035 | 1036 | 1037 | 1038 | 1039 | 1040 | 1041 | 1042 | 1043 | 1044 | 1045 | 1046 | 1047 | 1048 | 1049 | 1050 | 1051 | 1052 | 1053 | 1054 | 1055 | 1056 | 1057 | 1058 | 1059 | 1060 | 1061 | 1062 | 1063 | 1064 | 1065 | 1066 | 1067 | 1068 | 1069 | 1070 | 1071 | 1072 | 1073 | 1074 | 1075 | 1076 | 1077 | 1078 | 1079 | 1080 | 1081 | 1082 | 1083 | 1084 | 1085 | 1086 | 1087 | 1088 | 1089 | 1090 | 1091 | 1092 | 1093 | 1094 | 1095 | 1096 | 1097 | 1098 | 1099 | 1100 | 1101 | 1102 | 1103 | 1104 | 1105 | 1106 | 1107 | 1108 | 1109 | 1110 | 1111 | 1112 | 1113 | 1114 | 1115 | 1116 | 1117 | 1118 | 1119 | 1120 | 1121 | 1122 | 1123 | 1124 | 1125 | 1126 | 1127 | 1128 | 1129 | 1130 | 1131 | 1132 | 1133 | 1134 | 1135 | 1136 | 1137 | 1138 | 1139 | 1140 | 1141 | 1142 |
TitleSocial Science Natural Science EngineeringPaperCode
Drori et al.-Science Education-Paper-
SayCan--Robotics & Embodied AIPaperCode
Inner monologue--Robotics & Embodied AIPaperCode
Language-Planners--Robotics & Embodied AIPaperCode
Social SimulacraSocial Simulation--Paper-
TEPsychology --PaperCode
Out of OnePolitical Science and Economy--Paper-
LIBROCS&SE--Paper-
Blind JudgementJurisprudence--Paper-
HortonPolitical Science and Economy--Paper-
DECKARD--Robotics & Embodied AIPaperCode
Planner-Actor-Reporter--Robotics & Embodied AIPaper-
DEPS--Robotics & Embodied AIPaper-
RCI--CS&SEPaperCode
Generative AgentsSocial Simulation--PaperCode
SCG--CS&SEPaper-
IGLU--Civil EngineeringPaper-
IELLM--Industrial AutomationPaper-
ChemCrow-Document and Data Management;
Documentation, Data Managent;
Science Education
-Paper-
Boiko et al.-Document and Data Management;
Documentation, Data Managent;
Science Education
-Paper-
GPT4IA--Industrial AutomationPaperCode
Self-collaboration--CS&SEPaper-
E2WM--Robotics & Embodied AIPaperCode
Akata et al.Psychology --Paper-
Ziems et al.Psychology;
Political Science and Economy;
Research Assistant 		--Paper-
AgentVerseSocial Simulation--PaperCode
SmolModels--CS&SE-Code
TidyBot--Robotics & Embodied AIPaperCode
PET--Robotics & Embodied AIPaper-
Voyager--Robotics & Embodied AIPaperCode
GITM--Robotics & Embodied AIPaperCode
NLSOM-Science Education-Paper-
LLM4RL--Robotics & Embodied AIPaper-
GPT Engineer--CS&SE-Code
Grossman et al.-Experiment Assistant;
Science Education		-Paper-
SQL-PALM--CS&SEPaper-
REMEMBER--Robotics & Embodied AIPaper-
DemoGPT--CS&SE-Code
ChatlawJurisprudence--PaperCode
RestGPT--CS&SEPaperCode
Dialogue shaping--Robotics & Embodied AIPaper-
TaPA--Robotics & Embodied AIPaper-
Ma et al.Psychology --Paper-
Math Agents-Science Education-Paper-
SocialAI SchoolSocial Simulation--Paper-
Unified Agent--Robotics & Embodied AIPaper-
Wiliams et al.Social Simulation--Paper-
Li et al.Social Simulation--Paper-
S3Social Simulation--Paper-
Dialogue Shaping--Robotics & Embodied AIPaper-
RoCo--Robotics & Embodied AIPaperCode
Sayplan--Robotics & Embodied AIPaperCode
aiflows--CS & SEPaperCode
ToolLLM--CS&SEPaperCode
ChatDEV--CS&SEPaper-
Chao et al.Social Simulation--Paper-
AgentSimsSocial Simulation--PaperCode
ChatMOF-Document and Data Management;
Science Education		-Paper-
MetaGPT--CS&SEPaperCode
L2MAC--CS&SEPaperCode
Codehelp-Science EducationCS&SEPaper-
AutoGen-Science Education-Paper-
RAH--CS&SEPaper-
DB-GPT--CS&SEPaperCode
RecMind--CS&SEPaper-
ChatEDA--CS&SEPaper-
InteRecAgent--CS&SEPaper-
PentestGPT--CS&SEPaper-
Codehelp--CS&SEPaper-
ProAgent--Robotics & Embodied AIPaper-
MindAgent--Robotics & Embodied AIPaper-
LEO--Robotics & Embodied AIPaper-
JARVIS-1--Robotics & Embodied AIPaper-
CLOVA--CS&SEPaper-
AgentTrust-Social Simulation-PaperCode
embodied-agents--Robotics & Embodied AI-Code
AgentOccam--CS&SEPaper-
1143 | 1144 | * More papers can be found at [More comprehensive Summarization](#-more-comprehensive-summarization). 1145 | 1146 | ## 📊 Evaluation on LLM-based Autonomous Agent 1147 | 1148 | 1149 | 1150 | 1151 | 1152 | 1153 | 1154 | 1155 | 1156 | 1157 | 1158 | 1159 | 1160 | 1161 | 1162 | 1163 | 1164 | 1165 | 1166 | 1167 | 1168 | 1169 | 1170 | 1171 | 1172 | 1173 | 1174 | 1175 | 1176 | 1177 | 1178 | 1179 | 1180 | 1181 | 1182 | 1183 | 1184 | 1185 | 1186 | 1187 | 1188 | 1189 | 1190 | 1191 | 1192 | 1193 | 1194 | 1195 | 1196 | 1197 | 1198 | 1199 | 1200 | 1201 | 1202 | 1203 | 1204 | 1205 | 1206 | 1207 | 1208 | 1209 | 1210 | 1211 | 1212 | 1213 | 1214 | 1215 | 1216 | 1217 | 1218 | 1219 | 1220 | 1221 | 1222 | 1223 | 1224 | 1225 | 1226 | 1227 | 1228 | 1229 | 1230 | 1231 | 1232 | 1233 | 1234 | 1235 | 1236 | 1237 | 1238 | 1239 | 1240 | 1241 | 1242 | 1243 | 1244 | 1245 | 1246 | 1247 | 1248 | 1249 | 1250 | 1251 | 1252 | 1253 | 1254 | 1255 | 1256 | 1257 | 1258 | 1259 | 1260 | 1261 | 1262 | 1263 | 1264 | 1265 | 1266 | 1267 | 1268 | 1269 | 1270 | 1271 | 1272 | 1273 | 1274 | 1275 | 1276 | 1277 | 1278 | 1279 | 1280 | 1281 | 1282 | 1283 | 1284 | 1285 | 1286 | 1287 | 1288 | 1289 | 1290 | 1291 | 1292 | 1293 | 1294 | 1295 | 1296 | 1297 | 1298 | 1299 | 1300 | 1301 | 1302 | 1303 | 1304 | 1305 | 1306 | 1307 | 1308 | 1309 | 1310 | 1311 | 1312 | 1313 | 1314 | 1315 | 1316 | 1317 | 1318 | 1319 | 1320 | 1321 | 1322 | 1323 | 1324 | 1325 | 1326 | 1327 | 1328 | 1329 | 1330 | 1331 | 1332 | 1333 | 1334 | 1335 | 1336 | 1337 | 1338 | 1339 | 1340 | 1341 | 1342 | 1343 | 1344 | 1345 | 1346 | 1347 | 1348 | 1349 | 1350 | 1351 | 1352 | 1353 | 1354 | 1355 | 1356 | 1357 | 1358 | 1359 | 1360 | 1361 | 1362 | 1363 | 1364 | 1365 | 1366 | 1367 | 1368 | 1369 | 1370 | 1371 | 1372 | 1373 | 1374 | 1375 | 1376 | 1377 | 1378 | 1379 | 1380 | 1381 | 1382 | 1383 | 1384 | 1385 | 1386 | 1387 | 1388 | 1389 | 1390 | 1391 | 1392 | 1393 | 1394 | 1395 | 1396 | 1397 | 1398 | 1399 | 1400 | 1401 | 1402 | 1403 | 1404 | 1405 | 1406 | 1407 | 1408 | 1409 | 1410 | 1411 | 1412 | 1413 | 1414 | 1415 | 1416 | 1417 | 1418 | 1419 | 1420 | 1421 | 1422 | 1423 | 1424 | 1425 | 1426 | 1427 | 1428 | 1429 | 1430 | 1431 | 1432 | 1433 | 1434 | 1435 | 1436 | 1437 | 1438 | 1439 | 1440 | 1441 | 1442 | 1443 | 1444 | 1445 |
ModelSubjective Objective BenchmarkPaperCode
WebShop-Environment Simulation;
Multi-task Evaluation 		✓PaperCode
Social SimulacraHuman AnnotationSocial Evaluation-Paper-
TE-Social Evaluation-PaperCode
LIBRO-Software Testing-Paper-
ReAct-Environment Simulation✓PaperCode
Out of One, ManyTuring TestSocial Evaluation;
Multi-task Evaluation		-Paper-
DEPS-Environment Simulation✓Paper-
Jalil et al.-Software Testing-PaperCode
Reflexion-Environment Simulation;
Multi-task Evaluation		-PaperCode
IGLU-Environment Simulation✓Paper-
Generative Agents Human Annoation;
Turing Test		--PaperCode
ToolBenchHuman AnnoationMulti-task Evalution✓PaperCode
GITM-Environment Simulation✓PaperCode
Two-Failures-Multi-task Evalution-Paper-
Voyager-Environment Simulation✓PaperCode
SocKET-Social Evaluation;
Multi-task Evaluation 		✓Paper-
Mobile-Env-Environment Simulation;
Multi-task Evaluation		✓PaperCode
Clembench-Environment Simulation;
Multi-task Evaluation		✓PaperCode
Mind2Web-Environment Simulation;
Multi-task Evaluation		✓PaperCode
Dialop-Social Evaluation✓PaperCode
Feldt et al.-Software Testing-Paper-
CO-LLMHuman AnnoationEnvironment Simulation-PaperCode
TachikumaHuman AnnoationEnvironment Simulation✓Paper-
WebArena-Environment Simulation✓PaperCode
RocoBench-Environment Simulation;
Social Evaluation;
Multi-task Evaluation		✓PaperCode
AgentSims-Social Evaluation-PaperCode
AgentBench-Multi-task Evaluation✓PaperCode
BOLAA-Environment Simulation;
Multi-task Evaluation;
Software Testing		✓PaperCode
Gentopia-Isolated Reasoning;
Multi-task Evaluation 		✓PaperCode
EmotionBenchHuman Annotation-✓PaperCode
PTB-Software Testing ✓Paper-
MintBench-Multi-task Evaluation✓PaperCode
MindAgent-Environment Simulation;
Multi-task Evaluation		✓Paper-
JARVIS-1-Environment Simulation-Paper-
TimeCharacGPT Annotation-✓PaperCode
AppWorld-Environment Simulation✓PaperCode
1446 | 1447 | * More papers can be found at [More comprehensive Summarization](#-more-comprehensive-summarization). 1448 | 1449 |
1450 | 1451 | ## 🌐 More Comprehensive Summarization 1452 | 1453 | We are maintaining an [interactive table](https://abyssinian-molybdenum-f76.notion.site/237e9f7515d543c0922c74f4c3012a77?v=0a309e53d6454afcbe7a5a7e169be0f9&pvs=4) that contains more comprehensive papers related to LLM-based Agents. This table includes details such as tags, authors, publication date, and more, allowing you to sort, filter, and find the papers of interest to you. 1454 | ![Complete Table](assets/table.png) 1455 | 1456 | ## 👨‍👨‍👧‍👦 Maintainers 1457 | - Lei Wang@[Paitesanshi](https://github.com/Paitesanshi) 1458 | - Chen Ma@[Uily](https://github.com/Yilu114) 1459 | - Xueyang Feng@[XueyangFeng](https://github.com/XueyangFeng) 1460 | 1461 | ## 📚 Citation 1462 | If you find this survey useful, please cite our paper: 1463 | ``` 1464 | @misc{wang2023survey, 1465 | title={A Survey on Large Language Model based Autonomous Agents}, 1466 | author={Lei Wang and Chen Ma and Xueyang Feng and Zeyu Zhang and Hao Yang and Jingsen Zhang and Zhiyuan Chen and Jiakai Tang and Xu Chen and Yankai Lin and Wayne Xin Zhao and Zhewei Wei and Ji-Rong Wen}, 1467 | year={2023}, 1468 | eprint={2308.11432}, 1469 | archivePrefix={arXiv}, 1470 | primaryClass={cs.AI} 1471 | } 1472 | ``` 1473 | 1474 | 1475 | ## 💪 How to Contribute 1476 | If you have a paper or are aware of relevant research that should be incorporated, please contribute via pull requests, issues, email, or other suitable methods. 1477 | 1478 | 1479 | ## 🫡 Acknowledgement 1480 | We thank the following people for their valuable suggestions and contributions to this survey: 1481 | - Yifan Song[@Yifan-Song793](https://github.com/Yifan-Song793) 1482 | - Qichen Zhao[@Andrewzh112](https://github.com/Andrewzh112) 1483 | - Ikko E. Ashimine[@eltociear](https://github.com/eltociear) 1484 | 1485 | 1486 | ## 📧 Contact Us 1487 | If you have any questions or suggestions, please contact us via: 1488 | - Email: wanglei154@ruc.edu.cn, xu.chen@ruc.edu.cn 1489 | 1490 | 1491 | -------------------------------------------------------------------------------- /assets/app&eval.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Paitesanshi/LLM-Agent-Survey/c6503602926a11131d1bd5947b2a1612982e4e4e/assets/app&eval.png -------------------------------------------------------------------------------- /assets/architecture-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Paitesanshi/LLM-Agent-Survey/c6503602926a11131d1bd5947b2a1612982e4e4e/assets/architecture-1.png -------------------------------------------------------------------------------- /assets/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Paitesanshi/LLM-Agent-Survey/c6503602926a11131d1bd5947b2a1612982e4e4e/assets/architecture.png -------------------------------------------------------------------------------- /assets/capability.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Paitesanshi/LLM-Agent-Survey/c6503602926a11131d1bd5947b2a1612982e4e4e/assets/capability.png -------------------------------------------------------------------------------- /assets/planning.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Paitesanshi/LLM-Agent-Survey/c6503602926a11131d1bd5947b2a1612982e4e4e/assets/planning.png -------------------------------------------------------------------------------- /assets/table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Paitesanshi/LLM-Agent-Survey/c6503602926a11131d1bd5947b2a1612982e4e4e/assets/table.png -------------------------------------------------------------------------------- /assets/trend.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Paitesanshi/LLM-Agent-Survey/c6503602926a11131d1bd5947b2a1612982e4e4e/assets/trend.png --------------------------------------------------------------------------------