├── .DS_Store
├── .gitignore
├── LICENSE
├── README.md
├── action_attr.md
└── text.py
/.DS_Store:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/xinzhel/LLM-Agent-Survey/d3814c7f4ccb399efdd641d63f735fb069cbea68/.DS_Store
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/xinzhel/LLM-Agent-Survey/d3814c7f4ccb399efdd641d63f735fb069cbea68/.gitignore
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2023 Xinzhe Li
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
2 |
3 | A Reading List for LLM-Agents (Last Major Updated: 14 Mar 2025)
4 |
5 |
6 |
7 |
8 | Xinzhe Li
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
20 | ### This Repository vs. Others
21 | Our Github Repository follows the selection criteria below:
22 | - **Allowing Coherent Understanding**: They can be systematically categoried into the unified framework in my survey, according to the use of LLM-Profiled Roles (LMPRs).
23 | - A general survey (Accepted at CoLing 2025): [A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning](https://arxiv.org/abs/2406.05804)
24 |
25 |
26 | - **High Quality**: Papers are published on ICML, ICLR, NeurIPS, *ACL (including EMNLP), and COLING. Or unpublished papers contain useful analysis and insightful novelty
27 | - Unpublished papers are marked with 💡 and will be updated upon publication.
28 | - Paper Reviews: The paper links to OpenReview (if available) are alwasy given. I often learn much more from and resonate with many reviews about the papers and evaluate some rejected papers with the reviews. (That's why I always like NeurIPS/ICLR papers).
29 | - **Exhasutive Review on [Search Workflows](https://github.com/xinzhel/LLM-Search)**
30 | - A corresponding survey on TMLR 2025: [A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworks](https://openreview.net/forum?id=x9VQFjtOPS)
31 |
32 | > From Apr 2025, I will not actively update this repo since my recent research focuses on [LLM Inference via Search or LIS](https://github.com/xinzhel/LLM-Search). But I am sure that you can follow some actively updated repos below for the latest papers. Here is some reason for my transfer: 1) the LIS algorithms/workflows contain less heuristic, delibrate design, and human-aligned slow thinking while generally applicable. 2) with the limited energy as an individual researcher on inference-time workflows/agents, I do not have enough energy to discover the details of newly published work in this general field. Putting all the energy on LIS can help me maintain the quality of my work delivered to the community. Hopefully I can find a professional team later to work more ambitiously.
33 |
34 | Other Github Repositories summarize related papers with less constrained selection criteria:
35 | * [AGI-Edgerunners/LLM-Agents-Papers](https://github.com/AGI-Edgerunners/LLM-Agents-Papers?tab=readme-ov-file)
36 | * [zjunlp/LLMAgentPapers](https://github.com/zjunlp/LLMAgentPapers)
37 | * [Paitesanshi/LLM-Agent-Survey](https://github.com/Paitesanshi/LLM-Agent-Survey)
38 | * [woooodyy/llm-agent-paper-list](https://github.com/woooodyy/llm-agent-paper-list)
39 | * [Autonomous-Agents](https://github.com/tmgthb/Autonomous-Agents)
40 |
41 | Other Github Repositories summarize related papers focusing on specific perspectives:
42 | * [LLM-Search](https://github.com/xinzhel/LLM-Search)
43 | * [nuster1128/LLM_Agent_Memory_Survey](https://github.com/nuster1128/LLM_Agent_Memory_Survey) : Focus on memory
44 | * [teacherpeterpan/self-correction-llm-papers](https://github.com/teacherpeterpan/self-correction-llm-papers) : Focus on feedback learning (Self Correction)
45 | * [git-disl/awesome-LLM-game-agent-papers](https://github.com/git-disl/awesome-LLM-game-agent-papers) : Focus on gaming applications
46 |
47 |
48 | ## Table of Contents
49 | - [🎁 Surveys](#gift-surveys)
50 | - [🚀 Tool Use](#rocket-tool-use)
51 | - [🧠 Planning](#brain-planning)
52 | - [Base Workflows](#base-workflows)
53 | - [Search Workflows](#search-workflows)
54 | - [Decomposition](#decomposition)
55 | - [PDDL + Local Search](#pddl+local-search)
56 | - [Others](#others)
57 | - [🔄 Feedback Learning](#arrows_counterclockwise-feedback-learning)
58 | - [🧩 Composition](#jigsaw-composition)
59 | - [Planning + Feedback Learning](#planning--feedback-learning)
60 | - [Planning + Tool Use](#planning--tool-use)
61 | - [🌍 World Modeling](#world_map-world-modeling)
62 |
64 | - [📊 Benchmarks](#bar_chart-benchmarks)
65 | - [📝 Citation](#memo-citation)
66 |
67 |
68 | ## :gift: Surveys
69 | - **A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning**, CoLing 2025 [[paper]](https://arxiv.org/abs/2406.05804)
70 | - **A Survey on Large Language Model based Autonomous Agents**, Frontiers of Computer Science 2024 [[paper]](https://arxiv.org/abs/2308.11432) | [[code]](https://github.com/Paitesanshi/LLM-Agent-Survey)
71 | - **Augmented Language Models: a Survey**, TMLR [[paper]](https://openreview.net/forum?id=jh7wH2AzKK)
72 | - **Understanding the planning of LLM agents: A survey**, arXiv [[paper]](https://arxiv.org/abs/2402.02716) 💡
73 | - **The Rise and Potential of Large Language Model Based Agents: A Survey**, arxiv [[paper]](https://arxiv.org/abs/2309.07864) 💡
74 | - **A Survey on the Memory Mechanism of Large Language Model based Agents**, arxiv [[paper]](https://arxiv.org/abs/2404.13501) 💡
75 |
76 | ### :rocket: Tool Use
77 | - **ReAct: Synergizing Reasoning and Acting in Language Models**, ICLR 2023 [[paper](https://openreview.net/pdf?id=WE_vluYUL-X)]
78 | - **Toolformer: Language Models Can Teach Themselves to Use Tools**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2302.04761)]
79 | - **HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face**, NeurIPS 2023 [[paper](https://api.semanticscholar.org/CorpusID:257833781)]
80 | - **API-Bank: A Benchmark for Tool-Augmented LLMs**, EMNLP 2023 [[paper](https://api.semanticscholar.org/CorpusID:258179056)]
81 | - **ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.11554)]
82 | - **MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting**, ACL 2023 [[paper](https://aclanthology.org/2023.acl-short.130/)]
83 | - **ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models**, EMNLP 2023 [[paper](https://doi.org/10.48550/arXiv.2305.14323)]
84 | - **ART: Automatic multi-step reasoning and tool-use for large language models**, arXiv.2303.09014 [[paper](https://doi.org/10.48550/arXiv.2303.09014)] 💡
85 | - **TALM: Tool Augmented Language Models**, arXiv.2205.12255 [[paper](https://doi.org/10.48550/arXiv.2205.12255)] 💡
86 | - **On the Tool Manipulation Capability of Open-source Large Language Models**, arXiv.2305.16504 [[paper](https://doi.org/10.48550/arXiv.2305.16504)] 💡
87 | - **Large Language Models as Tool Makers**, arXiv.2305.17126 [[paper](https://doi.org/10.48550/arXiv.2305.17126)] 💡
88 | - **GEAR: Augmenting Language Models with Generalizable and Efficient Tool Resolution**, arXiv.2307.08775 [[paper](https://doi.org/10.48550/arXiv.2307.08775)] 💡
89 | - **ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs**, arXiv.2307.16789 [[paper](https://doi.org/10.48550/arXiv.2307.16789)] 💡
90 | - **Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models**, arXiv.2308.00675 [[paper](https://doi.org/10.48550/arXiv.2308.00675)] 💡
91 | - **MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback**, arXiv.2309.10691 [[paper](https://doi.org/10.48550/arXiv.2309.10691)] 💡
92 | - **Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning**, arXiv.2309.10814 [[paper](https://doi.org/10.48550/arXiv.2309.10814)] 💡
93 |
94 | ### :brain: Planning
95 |
96 | #### Base Workflows
97 | - **On the Planning Abilities of Large Language Models -- A Critical Investigation**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.15771)]
98 |
99 | #### Search Workflows
100 | Details in [the page (on the way to be publised)](https://github.com/xinzhel/LLM-Search).
101 | - **Alphazero-like Tree-Search can guide large language model decoding and training**, ICML 2024 [[paper](https://arxiv.org/abs/2309.17179)]
102 | + Search Algorithm: MCTS
103 | - **Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models**, ICML 2024 [[paper](https://openreview.net/forum?id=6LNTSrJjBe)]
104 | + Search Algorithm: MCTS
105 | - **When is Tree Search Useful for {LLM} Planning? It Depends on the Discriminator**, ACL 2024 [[paper](https://aclanthology.org/2024.acl-long.738/)]
106 | + Search Algorithm: MCTS
107 | - **Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation**, ACL findings 2024 [[paper](https://aclanthology.org/2024.findings-acl.95/)]
108 | + Search Algorithm: MCTS
109 | - **Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs**, ACL 2024 [[paper](https://aclanthology.org/2024.acl-long.665/)]
110 | + Search Algorithm: BFS/DFS
111 | - **LLM-A\*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning**, EMNLP findings 2024 [[paper](https://arxiv.org/pdf/2407.02511)] | [[code]](https://github.com/SilinMeng0510/llm-astar/)
112 | + Search Algorithm: A*
113 | - **LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models**, COLM2024 [[paper]](https://arxiv.org/abs/2404.05221) | [[code]](https://github.com/maitrix-org/llm-reasoners)
114 | - **Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models**, arXiv.2310.04406 [[paper](https://doi.org/10.48550/arXiv.2310.04406)] 💡
115 | - **Large Language Model Guided Tree-of-Thought**, arXiv.2305.08291 [[paper](https://doi.org/10.48550/arXiv.2305.08291)]💡
116 | - **Tree Search for Language Model Agents**, Under Review [[paper](https://openreview.net/forum?id=kpL66Mvd2a)]💡
117 | + Search Algorithm: Best-First Search
118 | - **Q\*: Improving multi-step reasoning for llms with deliberative planning**, Under Review [[paper](https://openreview.net/forum?id=F7QNwDYG6I)]💡
119 | + Search Algorithm: A*
120 | - **Planning with Large Language Models for Code Generation**, ICLR 2023 [[paper](ttps://openreview.net/forum?id=Lr8cOOtYbfL)]
121 | + Search Algorithm: MCTS
122 | - **Tree of Thoughts: Deliberate Problem Solving with Large Language Models**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.10601)]
123 | + Search Algorithm: BFS/DFS
124 | - **LLM-MCTS:Large Language Models as Commonsense Knowledge for Large-Scale Task Planning**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=tED747HURfX)] | [[code]](https://github.com/1989Ryan/llm-mcts)
125 | + Search Algorithm: MCTS
126 | - **Self-Evaluation Guided Beam Search for Reasoning**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=Bw82hwg5Q3)]
127 | + Search Algorithm: BFS/DFS
128 | - **PathFinder: Guided Search over Multi-Step Reasoning Paths**, NeurIPS 2023 R0-FoMo [[paper](https://openreview.net/forum?id=Bw82hwg5Q3)]
129 | + Search Algorithm: Beam Search
130 | - **Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts**, EMNLP 2023 [[paper](https://doi.org/10.48550/arXiv.2310.14628)]
131 | - **RAP: Reasoning with Language Model is Planning with World Model**, EMNLP 2023 [[paper](https://api.semanticscholar.org/CorpusID:258865812)]
132 | + Search Algorithm: MCTS
133 | - **Prompt-Based Monte-Carlo Tree Search for Goal-oriented Dialogue Policy Planning**, EMNLP 2023 [[paper](https://aclanthology.org/2023.emnlp-main.439/)]
134 | + Search Algorithm: MCTS
135 | - **Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Design**, EMNLP findings 2023 [[paper](https://aclanthology.org/2023.findings-emnlp.560/)]
136 | + Search Algorithm: MCTS
137 | - **Agent q: Advanced reasoning and learning for autonomous ai agents**, arXiv.2309.10814 [[paper](https://arxiv.org/abs/2408.07199)] 💡
138 | + Search Algorithm: MCTS
139 |
140 |
141 | #### Decomposition
142 | - **HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=yHdTscY6Ci)] | [[code]](https://github.com/microsoft/JARVIS/tree/main/hugginggpt)
143 | - **Least-to-Most Prompting Enables Complex Reasoning in Large Language Models**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=WZH7099tgfM)]
144 |
145 | #### PDDL+Local Search
146 | - **Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning**, NeurIPS 2023 [[paper](https://arxiv.org/abs/2305.14909)] | [[code]](https://github.com/GuanSuns/LLMs-World-Models-for-Planning)
147 | - **On the Planning Abilities of Large Language Models - A Critical Investigation**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=X6dEqXIsEW)] | [[code]](https://github.com/karthikv792/LLMs-Planning)
148 | - **PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=YXogl4uQUO)] | [[code]](https://github.com/karthikv792/LLMs-Planning)
149 |
150 | #### Others
151 | - **LLM+P: Empowering Large Language Models with Optimal Planning Proficiency**, arXiv.2304.11477 [[paper](https://doi.org/10.48550/arXiv.2304.11477)]💡
152 |
153 | ### :arrows_counterclockwise: Feedback Learning
154 | - **Reflexion: Language Agents with Verbal Reinforcement Learning**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2303.11366)]
155 | - **Self-Refine: Iterative Refinement with Self-Feedback**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2303.17651)]
156 | - **SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning**, ICLR 2024 [[paper]](https://arxiv.org/abs/2308.00436) | [[code]](https://github.com/ningmiao/selfcheck)
157 | - **Learning From Correctness Without Prompting Makes LLM Efficient Reasoner**, COLM2024 [[paper]](https://openreview.net/forum?id=dcbNzhVVQj#discussion)
158 | - **Learning From Mistakes Makes LLM Better Reasoner**, arXiv [[paper]](https://arxiv.org/abs/2310.20689) | [[code]](https://github.com/microsoft/LEMA)💡
159 | - **LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback** ACL 2024 [[paper]](https://arxiv.org/abs/2406.03363)
160 |
161 |
162 | ### :jigsaw: Composition
163 | #### Planning + Feedback Learning
164 | - **AdaPlanner: Adaptive Planning from Feedback with Language Models**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.16653)]
165 | - **CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing**, ICLR 2024 [[paper](https://openreview.net/forum?id=Sx038qxjek)]
166 | - **ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning**, arXiv.2308.13724 [[paper](https://doi.org/10.48550/arXiv.2308.13724)] 💡
167 |
168 | #### Planning + Tool Use
169 | - **ToolChain: Efficient Action Space Navigation in Large Language Models with A\* Search**, ICLR 2024 [[paper](https://openreview.net/forum?id=B6pQxqUcT8)]
170 | - **TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents**, FMDM @ NeurIPS 2023 [[paper](https://openreview.net/forum?id=GrkgKtOjaH)]
171 | - **TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems**, LLMAgents @ ICLR 2024 [[paper](https://doi.org/10.48550/arXiv.2311.11315)]
172 |
173 | ### :world_map: World Modeling
174 |
175 | - **Can Language Models Serve as Text-Based World Simulators?**, ACL 2024 [[paper](https://arxiv.org/abs/2406.06485)] | [[code]](https://github.com/cognitiveailab/GPT-simulator)
176 | - **Making Large Language Models into World Models with Precondition and Effect Knowledge**, arXiv [[paper](https://arxiv.org/abs/2409.12278)] 💡
177 |
178 | - **Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning**, NeurIPS 2023 [[paper](https://arxiv.org/abs/2305.14909)] | [[code]](https://github.com/GuanSuns/LLMs-World-Models-for-Planning)
179 | - **ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Games**, EMNLP 2023 [[paper](https://aclanthology.org/2023.emnlp-main.830/)] | [[code]](https://github.com/cognitiveailab/BYTESIZED32/tree/main)
180 |
181 | ### :bar_chart: Benchmarks
182 | #### Tool-Use Benchmarks
183 | - **MetaTool Benchmark: Deciding Whether to Use Tools and Which to Use**, arXiv.2310.03128 [[paper](https://doi.org/10.48550/arXiv.2310.03128)] 💡
184 | - **TaskBench: Benchmarking Large Language Models for Task Automation**, arXiv.2311.18760 [[paper](https://doi.org/10.48550/arXiv.2311.18760)] 💡
185 |
186 | #### Planning Benchmarks
187 | - **Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change)**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2206.10498)]
188 |
189 | ## :star: Star History
190 |
191 |
192 |
193 |
194 |
195 |
196 |
197 |
198 |
199 | ## :memo: Citation
200 |
201 | If you find our work helpful, you can cite this paper as:
202 |
203 | ```bibtex
204 | @inproceedings{li-2025-review,
205 | title = "A Review of Prominent Paradigms for {LLM}-Based Agents: Tool Use, Planning (Including {RAG}), and Feedback Learning",
206 | author = "Li, Xinzhe",
207 | editor = "Rambow, Owen and
208 | Wanner, Leo and
209 | Apidianaki, Marianna and
210 | Al-Khalifa, Hend and
211 | Eugenio, Barbara Di and
212 | Schockaert, Steven",
213 | booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
214 | month = jan,
215 | year = "2025",
216 | address = "Abu Dhabi, UAE",
217 | publisher = "Association for Computational Linguistics",
218 | url = "https://aclanthology.org/2025.coling-main.652/",
219 | pages = "9760--9779",
220 | }
221 | ```
222 |
223 | ```bibtex
224 | @article{li2025a,
225 | title={A Survey on {LLM} Test-Time Compute via Search: Tasks, {LLM} Profiling, Search Algorithms, and Relevant Frameworks},
226 | author={Xinzhe Li},
227 | journal={Transactions on Machine Learning Research},
228 | issn={2835-8856},
229 | year={2025},
230 | url={https://openreview.net/forum?id=x9VQFjtOPS},
231 | note={}
232 | }
233 | ```
234 |
--------------------------------------------------------------------------------
/action_attr.md:
--------------------------------------------------------------------------------
1 |
2 |
3 | ### Concise Definitions
4 |
5 | * **Constrained**: whether the action space is limited by predefined rules or operators (e.g., legal chess moves vs. unconstrained natural language).
6 | * **Heterogeneous**: whether actions come from *different types* (e.g., “click button,” “type text,” “scroll page”), as opposed to being uniform.
7 | * **State-Dependent**: whether the set of valid actions changes depending on the current environment state (e.g., you can only “pick up a mug” if one is present).
8 |
9 | ---
10 |
11 | ### Verification of Categorization
12 |
13 | 1. **Embodied Tasks**
14 |
15 | * Discrete ✓ (actions are predefined like “pick up,” “open door”)
16 | * Constrained: **Yes** (grammar of possible actions)
17 | * Heterogeneous: **Yes** (different types of interactions: pick, move, open)
18 | * State-Dependent: **Yes** (only available if object exists)
19 | * Action Reversible: **Maybe** (you can “drop” after “pick up,” but not always)
20 | * Execution: 🌐 Task-Essential
21 |
22 | 2. **Combinatorial Tasks (Game-of-24, Chess)**
23 |
24 | * Discrete ✓
25 | * Constrained: **Yes** (rules of math ops, chess moves)
26 | * Heterogeneous: **No** (all actions are uniform type: numbers/operators or chess moves)
27 | * State-Dependent: **Yes** (legal moves depend on current state)
28 | * Action Reversible: **Maybe** (some moves reversible, some not)
29 | * Execution: 🌐 Task-Essential
30 |
31 | 3. **Web Navigations (WebShop, WebArena)**
32 |
33 | * Discrete ✓
34 | * Constrained: **Yes** (set of clickable links, form options)
35 | * Heterogeneous: **Yes** (click, type, scroll, select)
36 | * State-Dependent: **Yes**
37 | * because available links/forms *do* change with navigation.
38 | * ⚠️: Some would argue “state-dependent = No,” since the set of UI elements usually exists regardless
39 | * Action Reversible: **Maybe** (e.g., can click back, but not always reversible)
40 | * Execution: 🌐 Task-Essential
41 |
42 |
43 | 4. **Graph Traversal**
44 |
45 | * Discrete ✓
46 | * Constrained: **Yes** (edges define legal moves)
47 | * Heterogeneous: **No** (all moves are “go to neighbor node”)
48 | * State-Dependent: **No** (set of edges from a node is fixed, not dynamic)
49 | * Action Reversible: **Maybe** (depends if traversal allows backtracking)
50 | * Execution: 🌐 Task-Essential
51 |
52 | 5. **Reasoning (Concatenation)**
53 |
54 | * Open text ✓
55 | * Constrained: **No** (free text generation)
56 | * Heterogeneous: **No** (always text)
57 | * State-Dependent: **No** (actions are unconstrained thoughts)
58 | * Action Reversible: **Yes** (in principle, can revise reasoning)
59 | * Execution: 📝 Internal
60 |
61 | 6. **Reasoning via QAs**
62 |
63 | * Open text ✓
64 | * Constrained: **No**
65 | * Heterogeneous: **No**
66 | * State-Dependent: **No**
67 | * Action Reversible: **Yes**
68 | * Execution: 📝 Internal
69 |
70 | 7. **Reasoning with Tool Invocation**
71 |
72 | * Open text + tool calls ✓
73 | * Constrained:
74 | * Text part: free-form, unconstrained (the LLM can output anything).
75 | * Tool calls: usually from a finite, predefined set of APIs (e.g., Calculator, Python.run, Search(query)), which is constrained.
76 | * Heterogeneous: **Yes** (mix of text and tool APIs)
77 | * State-Dependent: **No** (tools available independent of state, unless restricted)
78 | * Action Reversible: **Maybe** (depends on tool)
79 | * Execution: 🛠 Auxiliary External
80 |
81 | 8. **Reasoning over Knowledge Graph**
82 |
83 | * Discrete + triplets ✓
84 | * Constrained: **Yes** (relations and entities)
85 | * Heterogeneous: **Yes** (different relation types)
86 | * State-Dependent: **No** (triplets are fixed; graph doesn’t change during reasoning)
87 | * Action Reversible: **Yes** (can backtrack queries)
88 | * Execution: 🌐 Task-Essential
89 |
90 | 9. **Tool-based Tasks**
91 |
92 | * Discrete ✓
93 | * Constrained: **Yes** (set of allowed tools)
94 | * Heterogeneous: **Yes** (different APIs)
95 | * State-Dependent: **No**
96 | * Action Reversible: **Maybe** (depends on tool)
97 | * Execution: 🛠 Auxiliary External
98 |
99 | 10. **Code Generation**
100 |
101 | * Open text ✓
102 | * Constrained: **No** (unconstrained token generation)
103 | * Heterogeneous: **No** (all tokens are code)
104 | * State-Dependent: **No**
105 | * Action Reversible: **Yes** (can overwrite/edit code)
106 | * Execution: 📝 Internal
107 |
108 | 11. **Goal-oriented Dialog**
109 |
110 | * Discrete (intents) ✓
111 | * Constrained: **Yes** (limited intents)
112 | * Heterogeneous: **Yes** (different types of intents / utterances)
113 | * State-Dependent: **Yes** (valid intents depend on conversation history)
114 | * Action Reversible: **No** (dialogue acts cannot be undone)
115 | * Execution: 📝 Internal
116 |
117 | ---
118 |
--------------------------------------------------------------------------------
/text.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/xinzhel/LLM-Agent-Survey/d3814c7f4ccb399efdd641d63f735fb069cbea68/text.py
--------------------------------------------------------------------------------
14 | 
15 | 
16 | 
36 | * [zjunlp/LLMAgentPapers](https://github.com/zjunlp/LLMAgentPapers) 
37 | * [Paitesanshi/LLM-Agent-Survey](https://github.com/Paitesanshi/LLM-Agent-Survey) 
38 | * [woooodyy/llm-agent-paper-list](https://github.com/woooodyy/llm-agent-paper-list) 
39 | * [Autonomous-Agents](https://github.com/tmgthb/Autonomous-Agents) 
40 |
41 | Other Github Repositories summarize related papers focusing on specific perspectives:
42 | * [LLM-Search](https://github.com/xinzhel/LLM-Search) 
43 | * [nuster1128/LLM_Agent_Memory_Survey](https://github.com/nuster1128/LLM_Agent_Memory_Survey) 
: Focus on memory
44 | * [teacherpeterpan/self-correction-llm-papers](https://github.com/teacherpeterpan/self-correction-llm-papers) 
: Focus on feedback learning (Self Correction)
45 | * [git-disl/awesome-LLM-game-agent-papers](https://github.com/git-disl/awesome-LLM-game-agent-papers) 
: Focus on gaming applications
46 |
47 |
48 | ## Table of Contents
49 | - [🎁 Surveys](#gift-surveys)
50 | - [🚀 Tool Use](#rocket-tool-use)
51 | - [🧠 Planning](#brain-planning)
52 | - [Base Workflows](#base-workflows)
53 | - [Search Workflows](#search-workflows)
54 | - [Decomposition](#decomposition)
55 | - [PDDL + Local Search](#pddl+local-search)
56 | - [Others](#others)
57 | - [🔄 Feedback Learning](#arrows_counterclockwise-feedback-learning)
58 | - [🧩 Composition](#jigsaw-composition)
59 | - [Planning + Feedback Learning](#planning--feedback-learning)
60 | - [Planning + Tool Use](#planning--tool-use)
61 | - [🌍 World Modeling](#world_map-world-modeling)
62 |
64 | - [📊 Benchmarks](#bar_chart-benchmarks)
65 | - [📝 Citation](#memo-citation)
66 |
67 |
68 | ## :gift: Surveys
69 | - **A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning**, CoLing 2025 [[paper]](https://arxiv.org/abs/2406.05804)
70 | - **A Survey on Large Language Model based Autonomous Agents**, Frontiers of Computer Science 2024 [[paper]](https://arxiv.org/abs/2308.11432) | [[code]](https://github.com/Paitesanshi/LLM-Agent-Survey)
71 | - **Augmented Language Models: a Survey**, TMLR [[paper]](https://openreview.net/forum?id=jh7wH2AzKK)
72 | - **Understanding the planning of LLM agents: A survey**, arXiv [[paper]](https://arxiv.org/abs/2402.02716) 💡
73 | - **The Rise and Potential of Large Language Model Based Agents: A Survey**, arxiv [[paper]](https://arxiv.org/abs/2309.07864) 💡
74 | - **A Survey on the Memory Mechanism of Large Language Model based Agents**, arxiv [[paper]](https://arxiv.org/abs/2404.13501) 💡
75 |
76 | ### :rocket: Tool Use
77 | - **ReAct: Synergizing Reasoning and Acting in Language Models**, ICLR 2023 [[paper](https://openreview.net/pdf?id=WE_vluYUL-X)]
78 | - **Toolformer: Language Models Can Teach Themselves to Use Tools**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2302.04761)]
79 | - **HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face**, NeurIPS 2023 [[paper](https://api.semanticscholar.org/CorpusID:257833781)]
80 | - **API-Bank: A Benchmark for Tool-Augmented LLMs**, EMNLP 2023 [[paper](https://api.semanticscholar.org/CorpusID:258179056)]
81 | - **ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.11554)]
82 | - **MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting**, ACL 2023 [[paper](https://aclanthology.org/2023.acl-short.130/)]
83 | - **ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models**, EMNLP 2023 [[paper](https://doi.org/10.48550/arXiv.2305.14323)]
84 | - **ART: Automatic multi-step reasoning and tool-use for large language models**, arXiv.2303.09014 [[paper](https://doi.org/10.48550/arXiv.2303.09014)] 💡
85 | - **TALM: Tool Augmented Language Models**, arXiv.2205.12255 [[paper](https://doi.org/10.48550/arXiv.2205.12255)] 💡
86 | - **On the Tool Manipulation Capability of Open-source Large Language Models**, arXiv.2305.16504 [[paper](https://doi.org/10.48550/arXiv.2305.16504)] 💡
87 | - **Large Language Models as Tool Makers**, arXiv.2305.17126 [[paper](https://doi.org/10.48550/arXiv.2305.17126)] 💡
88 | - **GEAR: Augmenting Language Models with Generalizable and Efficient Tool Resolution**, arXiv.2307.08775 [[paper](https://doi.org/10.48550/arXiv.2307.08775)] 💡
89 | - **ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs**, arXiv.2307.16789 [[paper](https://doi.org/10.48550/arXiv.2307.16789)] 💡
90 | - **Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models**, arXiv.2308.00675 [[paper](https://doi.org/10.48550/arXiv.2308.00675)] 💡
91 | - **MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback**, arXiv.2309.10691 [[paper](https://doi.org/10.48550/arXiv.2309.10691)] 💡
92 | - **Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning**, arXiv.2309.10814 [[paper](https://doi.org/10.48550/arXiv.2309.10814)] 💡
93 |
94 | ### :brain: Planning
95 |
96 | #### Base Workflows
97 | - **On the Planning Abilities of Large Language Models -- A Critical Investigation**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.15771)]
98 |
99 | #### Search Workflows
100 | Details in [the page (on the way to be publised)](https://github.com/xinzhel/LLM-Search).
101 | - **Alphazero-like Tree-Search can guide large language model decoding and training**, ICML 2024 [[paper](https://arxiv.org/abs/2309.17179)]
102 | + Search Algorithm: MCTS
103 | - **Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models**, ICML 2024 [[paper](https://openreview.net/forum?id=6LNTSrJjBe)]
104 | + Search Algorithm: MCTS
105 | - **When is Tree Search Useful for {LLM} Planning? It Depends on the Discriminator**, ACL 2024 [[paper](https://aclanthology.org/2024.acl-long.738/)]
106 | + Search Algorithm: MCTS
107 | - **Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation**, ACL findings 2024 [[paper](https://aclanthology.org/2024.findings-acl.95/)]
108 | + Search Algorithm: MCTS
109 | - **Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs**, ACL 2024 [[paper](https://aclanthology.org/2024.acl-long.665/)]
110 | + Search Algorithm: BFS/DFS
111 | - **LLM-A\*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning**, EMNLP findings 2024 [[paper](https://arxiv.org/pdf/2407.02511)] | [[code]](https://github.com/SilinMeng0510/llm-astar/)
112 | + Search Algorithm: A*
113 | - **LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models**, COLM2024 [[paper]](https://arxiv.org/abs/2404.05221) | [[code]](https://github.com/maitrix-org/llm-reasoners)
114 | - **Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models**, arXiv.2310.04406 [[paper](https://doi.org/10.48550/arXiv.2310.04406)] 💡
115 | - **Large Language Model Guided Tree-of-Thought**, arXiv.2305.08291 [[paper](https://doi.org/10.48550/arXiv.2305.08291)]💡
116 | - **Tree Search for Language Model Agents**, Under Review [[paper](https://openreview.net/forum?id=kpL66Mvd2a)]💡
117 | + Search Algorithm: Best-First Search
118 | - **Q\*: Improving multi-step reasoning for llms with deliberative planning**, Under Review [[paper](https://openreview.net/forum?id=F7QNwDYG6I)]💡
119 | + Search Algorithm: A*
120 | - **Planning with Large Language Models for Code Generation**, ICLR 2023 [[paper](ttps://openreview.net/forum?id=Lr8cOOtYbfL)]
121 | + Search Algorithm: MCTS
122 | - **Tree of Thoughts: Deliberate Problem Solving with Large Language Models**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.10601)]
123 | + Search Algorithm: BFS/DFS
124 | - **LLM-MCTS:Large Language Models as Commonsense Knowledge for Large-Scale Task Planning**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=tED747HURfX)] | [[code]](https://github.com/1989Ryan/llm-mcts)
125 | + Search Algorithm: MCTS
126 | - **Self-Evaluation Guided Beam Search for Reasoning**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=Bw82hwg5Q3)]
127 | + Search Algorithm: BFS/DFS
128 | - **PathFinder: Guided Search over Multi-Step Reasoning Paths**, NeurIPS 2023 R0-FoMo [[paper](https://openreview.net/forum?id=Bw82hwg5Q3)]
129 | + Search Algorithm: Beam Search
130 | - **Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts**, EMNLP 2023 [[paper](https://doi.org/10.48550/arXiv.2310.14628)]
131 | - **RAP: Reasoning with Language Model is Planning with World Model**, EMNLP 2023 [[paper](https://api.semanticscholar.org/CorpusID:258865812)]
132 | + Search Algorithm: MCTS
133 | - **Prompt-Based Monte-Carlo Tree Search for Goal-oriented Dialogue Policy Planning**, EMNLP 2023 [[paper](https://aclanthology.org/2023.emnlp-main.439/)]
134 | + Search Algorithm: MCTS
135 | - **Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Design**, EMNLP findings 2023 [[paper](https://aclanthology.org/2023.findings-emnlp.560/)]
136 | + Search Algorithm: MCTS
137 | - **Agent q: Advanced reasoning and learning for autonomous ai agents**, arXiv.2309.10814 [[paper](https://arxiv.org/abs/2408.07199)] 💡
138 | + Search Algorithm: MCTS
139 |
140 |
141 | #### Decomposition
142 | - **HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=yHdTscY6Ci)] | [[code]](https://github.com/microsoft/JARVIS/tree/main/hugginggpt)
143 | - **Least-to-Most Prompting Enables Complex Reasoning in Large Language Models**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=WZH7099tgfM)]
144 |
145 | #### PDDL+Local Search
146 | - **Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning**, NeurIPS 2023 [[paper](https://arxiv.org/abs/2305.14909)] | [[code]](https://github.com/GuanSuns/LLMs-World-Models-for-Planning)
147 | - **On the Planning Abilities of Large Language Models - A Critical Investigation**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=X6dEqXIsEW)] | [[code]](https://github.com/karthikv792/LLMs-Planning)
148 | - **PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change**, NeurIPS 2023 [[paper](https://openreview.net/forum?id=YXogl4uQUO)] | [[code]](https://github.com/karthikv792/LLMs-Planning)
149 |
150 | #### Others
151 | - **LLM+P: Empowering Large Language Models with Optimal Planning Proficiency**, arXiv.2304.11477 [[paper](https://doi.org/10.48550/arXiv.2304.11477)]💡
152 |
153 | ### :arrows_counterclockwise: Feedback Learning
154 | - **Reflexion: Language Agents with Verbal Reinforcement Learning**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2303.11366)]
155 | - **Self-Refine: Iterative Refinement with Self-Feedback**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2303.17651)]
156 | - **SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning**, ICLR 2024 [[paper]](https://arxiv.org/abs/2308.00436) | [[code]](https://github.com/ningmiao/selfcheck)
157 | - **Learning From Correctness Without Prompting Makes LLM Efficient Reasoner**, COLM2024 [[paper]](https://openreview.net/forum?id=dcbNzhVVQj#discussion)
158 | - **Learning From Mistakes Makes LLM Better Reasoner**, arXiv [[paper]](https://arxiv.org/abs/2310.20689) | [[code]](https://github.com/microsoft/LEMA)💡
159 | - **LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback** ACL 2024 [[paper]](https://arxiv.org/abs/2406.03363)
160 |
161 |
162 | ### :jigsaw: Composition
163 | #### Planning + Feedback Learning
164 | - **AdaPlanner: Adaptive Planning from Feedback with Language Models**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2305.16653)]
165 | - **CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing**, ICLR 2024 [[paper](https://openreview.net/forum?id=Sx038qxjek)]
166 | - **ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning**, arXiv.2308.13724 [[paper](https://doi.org/10.48550/arXiv.2308.13724)] 💡
167 |
168 | #### Planning + Tool Use
169 | - **ToolChain: Efficient Action Space Navigation in Large Language Models with A\* Search**, ICLR 2024 [[paper](https://openreview.net/forum?id=B6pQxqUcT8)]
170 | - **TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents**, FMDM @ NeurIPS 2023 [[paper](https://openreview.net/forum?id=GrkgKtOjaH)]
171 | - **TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems**, LLMAgents @ ICLR 2024 [[paper](https://doi.org/10.48550/arXiv.2311.11315)]
172 |
173 | ### :world_map: World Modeling
174 |
175 | - **Can Language Models Serve as Text-Based World Simulators?**, ACL 2024 [[paper](https://arxiv.org/abs/2406.06485)] | [[code]](https://github.com/cognitiveailab/GPT-simulator)
176 | - **Making Large Language Models into World Models with Precondition and Effect Knowledge**, arXiv [[paper](https://arxiv.org/abs/2409.12278)] 💡
177 |
178 | - **Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning**, NeurIPS 2023 [[paper](https://arxiv.org/abs/2305.14909)] | [[code]](https://github.com/GuanSuns/LLMs-World-Models-for-Planning)
179 | - **ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Games**, EMNLP 2023 [[paper](https://aclanthology.org/2023.emnlp-main.830/)] | [[code]](https://github.com/cognitiveailab/BYTESIZED32/tree/main)
180 |
181 | ### :bar_chart: Benchmarks
182 | #### Tool-Use Benchmarks
183 | - **MetaTool Benchmark: Deciding Whether to Use Tools and Which to Use**, arXiv.2310.03128 [[paper](https://doi.org/10.48550/arXiv.2310.03128)] 💡
184 | - **TaskBench: Benchmarking Large Language Models for Task Automation**, arXiv.2311.18760 [[paper](https://doi.org/10.48550/arXiv.2311.18760)] 💡
185 |
186 | #### Planning Benchmarks
187 | - **Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change)**, NeurIPS 2023 [[paper](https://doi.org/10.48550/arXiv.2206.10498)]
188 |
189 | ## :star: Star History
190 |
191 |
192 |