└── README.md /README.md: -------------------------------------------------------------------------------- 1 |
2 |

Awesome LLM RAG

3 | 4 |
5 | 6 | \ 7 | [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/hee9joon/Awesome-Diffusion-Models) 8 | [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT) 9 | [![Made With Love](https://img.shields.io/badge/Made%20With-Love-red.svg)](https://github.com/chetanraj/awesome-github-badges) 10 | 11 | 12 | 13 | This repo aims to record advanced papers on Retrieval Augmented Generation (RAG) in LLMs. 14 | 15 | We strongly encourage the researchers that want to promote their fantastic work to the LLM RAG to make pull request to update their paper's information! 16 | 17 | 18 | --- 19 | 20 | ## Contents 21 | 22 | - [Resources](#resources) 23 | - [Workshops and Tutorials](#workshops-and-tutorials) 24 | - [Papers](#papers) 25 | - [Survey and Benchmark](#survey-and-benchmark) 26 | - [Retrieval-enhanced LLMs](#retrieval-enhanced-llms) 27 | - [RAG Instruction Tuning](#rag-instruction-tuning) 28 | - [RAG In-Context Learning](#rag-in-context-learning) 29 | - [RAG Embeddings](#rag-embeddings) 30 | - [RAG Simulators](#rag-simulators) 31 | - [RAG Search](#rag-search) 32 | - [RAG Long-text and Memory](#rag-long-text-and-memory) 33 | - [RAG Evaluation](#rag-evaluation) 34 | - [RAG Optimization](#rag-optimization) 35 | - [RAG Application](#rag-application) 36 | 37 | 38 | 39 | --- 40 | 41 | # Resources 42 | 43 | ## Workshops and Tutorials 44 | 45 | **Personalized Generative AI** 46 | *Zheng Chen, Ziyan Jiang, Fan Yang, Zhankui He, Yupeng Hou, Eunah Cho, Julian McAuley, Aram Galstyan, Xiaohua Hu, Jie Yang* 47 | CIKM 23 – Oct 2023 [[link](https://sites.google.com/view/pgai2023/home)] 48 | 49 | **First Workshop on Recommendation with Generative Models** 50 | *Wenjie Wang, Yong Liu, Yang Zhang, Weiwen Liu, Fuli Feng, Xiangnan He, Aixin Sun* 51 | CIKM 23 – Oct 2023 [[link](https://rgm-cikm23.github.io/)] 52 | 53 | **First Workshop on Generative Information Retrieval** 54 | *Gabriel Bénédict, Ruqing Zhang, Donald Metzler* 55 | SIGIR 23 – Jul 2023 [[link](https://coda.io/@sigir/gen-ir)] 56 | 57 | **Retrieval-based Language Models and Applications** 58 | *Akari Asai, Sewon Min, Zexuan Zhong, Danqi Chen* 59 | ACL 23 – Jul 2023 [[link](https://acl2023-retrieval-lm.github.io/)] 60 | 61 | **Become a Generative AI Developer** 62 | *Richie Cotton, Olivier Mertens, Korey Stegared-Pace, James Briggs, Vincent Vankrunkelsven, Alara Dirik, Jacob Marquez, Priyanka Asnani* 63 | DataCamp [[link](https://www.datacamp.com/ai-code-alongs)] 64 | 65 | # Papers 66 | 67 | ## Survey and Benchmark 68 | 69 | **Benchmarking Large Language Models in Retrieval-Augmented Generation** \ 70 | *Jiawei Chen, Hongyu Lin, Xianpei Han, Le Sun* \ 71 | arXiv 2023. [[Paper](https://arxiv.org/abs/2309.01431)][[Github](https://github.com/chen700564/RGB)] \ 72 | 4 Sep 2023 73 | 74 | 75 | 76 | ## Retrieval-enhanced LLMs 77 | 78 | 83 | 84 | **Adaptive Retrieval without Self-Knowledge? Bringing Uncertainty Back Home** \ 85 | Viktor Moskvoretskii, Maria Lysyuk, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, Alexander Panchenko \ 86 | arxiv – Jan 2025 [[paper](https://arxiv.org/abs/2501.12835)] 87 | 88 | **DFA-RAG: Conversational Semantic Router for Large Language Model with Definite Finite Automaton** \ 89 | Yiyou Sun, Junjie Hu, Wei Cheng, Haifeng Chen \ 90 | ICML 24 – Feb 2024 [[paper](https://arxiv.org/abs/2402.04411)] 91 | 92 | **Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models** \ 93 | *Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, Dong Yu* \ 94 | arxiv - Nov 2023 [[Paper](https://arxiv.org/abs/2311.09210)] 95 | 96 | **REST: Retrieval-Based Speculative Decoding** \ 97 | *Zhenyu He, Zexuan Zhong, Tianle Cai, Jason D Lee, Di He* \ 98 | arXiv - Nov 2023 [[Paper](https://arxiv.org/abs/2311.08252)][[Github](https://github.com/fasterdecoding/rest)] 99 | 100 | 101 | **Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection** 102 | *Anonymous* 103 | ICLR 24 – Oct 2023 [[paper](https://openreview.net/forum?id=hSyW5go0v8)] 104 | 105 | **Self-Knowledge Guided Retrieval Augmentation for Large Language Models** \ 106 | *Yile Wang, Peng Li, Maosong Sun, Yang Liu* \ 107 | arXiv - Oct 2023 [[Ppaer](https://arxiv.org/abs/2310.05002)] 108 | 109 | 110 | **Retrieval meets Long Context Large Language Models** \ 111 | *Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, Bryan Catanzaro* \ 112 | arxiv - Oct 2023 [[Paper](https://arxiv.org/abs/2310.03025)] 113 | 114 | 115 | **DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines** 116 | *Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, Christopher Potts* 117 | arXiv – Oct 2023 [[paper](https://arxiv.org/abs/2310.03714)] [[code](https://github.com/stanfordnlp/dspy)] 118 | 119 | **Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts** 120 | *Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, Yu Su* 121 | ICLR 24 – May 2023 [[paper](https://arxiv.org/abs/2305.13300)] [[code](https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict)] 122 | 123 | **Active Retrieval Augmented Generation** 124 | *Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, Graham Neubig* 125 | arXiv – May 2023 [[paper](https://arxiv.org/abs/2305.06983)] [[code](https://github.com/jzbjyb/FLARE)] 126 | 127 | 128 | **REPLUG: Retrieval-Augmented Black-Box Language Models** 129 | *Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih* 130 | arXiv – Jan 2023 [[paper](https://arxiv.org/abs/2301.12652)] 131 | 132 | 133 | **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks** 134 | *Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela* 135 | NeurIPS 2020 - May 2020 [[Paper](https://arxiv.org/abs/2005.11401)] 136 | 137 | 138 | 139 | ## RAG Instruction Tuning 140 | 141 | **RA-DIT: Retrieval-Augmented Dual Instruction Tuning** 142 | *Anonymous* 143 | ICLR 24 – Oct 23 [[paper](https://openreview.net/forum?id=22OTbutug9)] 144 | 145 | **InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining** 146 | *Boxin Wang, Wei Ping, Lawrence McAfee, Peng Xu, Bo Li, Mohammad Shoeybi, Bryan Catanzaro* \ 147 | arXiv - Oct 23 [[paper](https://openreview.net/forum?id=4stB7DFLp6)] 148 | 149 | 150 | ## RAG In-Context Learning 151 | 152 | **In-Context Retrieval-Augmented Language Models** 153 | *Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham* 154 | AI21 Labs – Jan 2023 [[paper](https://uploads-ssl.webflow.com/60fd4503684b466578c0d307/63c6c20dec4479564db21819_NEW_In_Context_Retrieval_Augmented_Language_Models.pdf)] [[code](https://github.com/AI21Labs/in-context-ralm)] 155 | 156 | 157 | ## RAG Embeddings 158 | 159 | **RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling** \ 160 | *Jingcheng Deng, Liang Pang, Huawei Shen, Xueqi Cheng* \ 161 | EMNLP 2023 - Oct 2023 [[Paper](https://arxiv.org/abs/2310.10567)][[Github](https://github.com/TrustedLLM/RegaVAE)] 162 | 163 | **Text Embeddings Reveal (Almost) As Much As Text** \ 164 | *John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander M. Rush* \ 165 | EMNLP 2023 - Oct 2023 [[Paper](https://arxiv.org/abs/2310.06816?ref=upstract.com)][[Github](https://github.com/jxmorris12/vec2text)] 166 | 167 | **Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents** \ 168 | *Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, Han Xiao* \ 169 | arXiv - Oct 2023. [[Paper](https://arxiv.org/abs/2310.19923)][[Model](https://huggingface.co/jinaai/jina-embeddings-v2-small-en)] 170 | 171 | ## RAG Simulators 172 | 173 | **KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants** \ 174 | *Kaustubh D. Dhole* \ 175 | Simulation of Conversational Intelligence in Chat, EACL 2024 [[Paper](https://arxiv.org/abs/2401.16454)] 176 | 177 | ## RAG Search 178 | 179 | ## RAG Long-text and Memory 180 | 181 | **HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models** \ 182 | *Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, Yu Su* \ 183 | arXiv - May 2024 [[paper](https://arxiv.org/abs/2405.14831)] [[GitHub](https://github.com/OSU-NLP-Group/HippoRAG)] 184 | 185 | **Understanding Retrieval Augmentation for Long-Form Question Answering** \ 186 | *Hung-Ting Chen, Fangyuan Xu, Shane A. Arora, Eunsol Choi* \ 187 | arXiv - Oct 2023 [[Paper](https://arxiv.org/abs/2310.12150)] 188 | 189 | ## RAG Evaluation 190 | 191 | **ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems** \ 192 | *Jon Saad-Falcon, Omar Khattab, Christopher Potts, Matei Zaharia* \ 193 | arXiv - Nov 2023. [[Paper](https://arxiv.org/abs/2311.09476)] [[Github](https://github.com/stanford-futuredata/ares)] 194 | 195 | 196 | 197 | ## RAG Optimization 198 | 199 | **Learning to Filter Context for Retrieval-Augmented Generation** \ 200 | *Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, Graham Neubig* \ 201 | arxiv- Nov 2023 [[Paper](https://arxiv.org/abs/2311.08377)][[Github](https://github.com/zorazrw/filco)] 202 | 203 | 204 | **Large Language Models Can Be Easily Distracted by Irrelevant Context** \ 205 | *Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, Denny Zhou* \ 206 | ICML 2023 - Jan 2023 [[Paper](https://arxiv.org/abs/2302.00093)][[Github](https://github.com/google-research-datasets/GSM-IC)] 207 | 208 | 209 | **Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks** \ 210 | *Akari Asai, Matt Gardner, Hannaneh Hajishirzi* \ 211 | NAACL 2022 - Dec 2021 [[Paper](https://arxiv.org/abs/2112.08688)][[Github](https://github.com/akariasai/evidentiality_qa)] 212 | 213 | **When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories** \ 214 | *Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, Hannaneh Hajishirzi* \ 215 | ACL 2023 - Dec 2022 [[Paper](https://arxiv.org/abs/2212.10511)][[Github](https://github.com/alextmallen/adaptive-retrieval)] 216 | 217 | 218 | 219 | ## RAG Application 220 | 221 | **Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination** \ 222 | *Haoqiang Kang, Xiao-Yang Liu* \ 223 | arXiv - Nov 2023 [[Paper](https://arxiv.org/abs/2311.15548)] 224 | 225 | 226 | **Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literature** \ 227 | *Alejandro Lozano, Scott L Fleming, Chia-Chun Chiang, Nigam Shah* \ 228 | arXiv - Oct 2023. [[Paper](https://arxiv.org/abs/2310.16146v1)] 229 | 230 | **PEARL: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers** \ 231 | *Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang, Steve Menezes, Tina Baghaee, Emmanuel Barajas Gonzalez, Jennifer Neville, Tara Safavi* \ 232 | arXiv - Nov 2023. [[Paper](https://arxiv.org/abs/2311.09180)] 233 | 234 | 235 | 236 | --------------------------------------------------------------------------------