└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # πŸš€ [ACM Computing Surveys 2025] Awesome Lifelong Learning Methods for Large Language Models (Updated Regularly; Latest Papers from NIPS2024, EMNLP2024, COLING2025, AAAI2025, and ICLR2025) 2 | 3 | [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) [![arXiv](https://img.shields.io/badge/arXiv-lifelong_LLM-b31b1b.svg)](https://arxiv.org/pdf/2406.06391) 4 | 5 | ## πŸ“’ Latest News 6 | - **2025.06** - We are excited to release the first benchmark [LifelongAgentBench](https://caixd-220529.github.io/LifelongAgentBench/) for lifelong learning of LLM Agents. The paper, source code, datasets are all available! 7 | - **2025.02** – Our survey **"Towards Lifelong Learning of Large Language Models: A Survey"** is accepted by ACM Computing Surveys! πŸ™Œ 8 | - **2025.01** – The latest research from **NIPS 2024, EMNLP 2024, COLING 2025, AAAI 2025, and ICLR 2025** is now available! Stay updated with the newest advancements in the field. 9 | - **2024.12** – Our survey **"Towards Lifelong Learning of Large Language Models: A Survey"** has been featured on [ζœΊε™¨δΉ‹εΏƒ](https://mp.weixin.qq.com/s/hilVDi-HAac_1DKvv8jv8Q) and [ηŸ₯乎](https://zhuanlan.zhihu.com/p/3352669117)! Check out these platforms for an in-depth interpretation. 10 | - **2024.06** – We have officially released our survey paper **"Towards Lifelong Learning of Large Language Models: A Survey"**! This repository is now live alongside the paperβ€”feel free to contribute by opening pull requests to add your papers! πŸš€ 11 | 12 | 13 | ## Introduction πŸ“œ 14 | This repository collects awesome surveys, resources, and papers for **Lifelong Learning** with **Large Language Models**. 15 | We define 12 lifelong learning scenarios as follows. Please refer to [this survey](https://arxiv.org/abs/2406.06391) for a detailed introduction. 16 | ![image](https://github.com/user-attachments/assets/2d448584-2225-4ef2-9dc8-c94025958f83) 17 | 18 | Additionally, you can refer to [this repository](https://github.com/qianlima-lab/awesome-lifelong-llm-agent) for **lifelong learning of LLM agents**. 19 | 20 | ## Survey πŸ“š 21 | - [Lifelong Learning of Large Language Model-based Agents: A Roadmap](https://arxiv.org/abs/2501.07278) (arXiv 2025.01) 22 | - [Towards Lifelong Learning of Large Language Models: A Survey](https://arxiv.org/abs/2406.06391) (arXiv 2024.06) 23 | - [Recent Advances of Foundation Language Models-based Continual Learning: A Survey](https://arxiv.org/abs/2405.18653) (arXiv 2024.05) 24 | - [Continual Learning of Large Language Models: A Comprehensive Survey](https://arxiv.org/abs/2404.16789) (arXiv 2024.04) 25 | - [Towards Incremental Learning in Large Language Models: A Critical Review](https://arxiv.org/abs/2404.18311) (arXiv 2024.04) 26 | - [Continual Learning for Large Language Models: A Survey](https://arxiv.org/abs/2402.01364) (arXiv 2024.02) 27 | - [Continual Learning with Pre-Trained Models: A Survey](https://arxiv.org/abs/2401.16386) (arXiv 2024.01) 28 | - [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://ieeexplore.ieee.org/abstract/document/10444954) (TPAMI 2024) 29 | - [How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances](https://aclanthology.org/2023.emnlp-main.516/) (EMNLP 2023) 30 | - [Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need](https://arxiv.org/abs/2303.07338) (arXiv 2023.03) 31 | - [Deep Class-Incremental Learning: A Survey](https://arxiv.org/abs/2302.03648) (arXiv 2023.03) 32 | - [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://arxiv.org/abs/2302.00487) (arXiv 2023.02) 33 | - [Continual Learning of Natural Language Processing Tasks: A Survey](https://arxiv.org/abs/2211.12701) (arXiv 2022.11) 34 | - [Continual Lifelong Learning in Natural Language Processing: A Survey](https://aclanthology.org/2020.coling-main.574/) (COLING 2020) 35 | 36 | ## Codebase πŸ–₯️ 37 | - [Codebase for Incremental Learning with LLMs](https://github.com/zzz47zzz/codebase-for-incremental-learning-with-llm) 38 | - [UIC-Liu-Lab ContinualLM](https://github.com/UIC-Liu-Lab/ContinualLM) 39 | - [Mammoth](https://github.com/aimagelab/mammoth) 40 | - [RevisitingCIL](https://github.com/zhoudw-zdw/RevisitingCIL) 41 | - [PyCIL](https://github.com/G-U-N/PyCIL) 42 | - [PyContinual](https://github.com/ZixuanKe/PyContinual) 43 | 44 | ## πŸ“„ Related Papers 45 | 46 | ### Keywords 🏷️ 47 | 48 | ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) **Continual Vertical Domain Pretraining** 49 | ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) **Continual Language Domain Pretraining** 50 | ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) **Continual Temporal Domain Pretraining** 51 | ![](https://img.shields.io/badge/Continual_Text_Classification-green) **Continual Text Classification** 52 | ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) **Continual Named Entity Recognition** 53 | ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) **Continual Relation Extraction** 54 | ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) **Continual Machine Translation** 55 | ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) **Continual Instruction-Tuning** 56 | ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) **Continual Knowledge Editing** 57 | ![](https://img.shields.io/badge/Continual_Alignment-lightgray) **Continual Alignment** 58 | ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) **Retrieval-Based Lifelong Learning** 59 | ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) **Tool-Based Lifelong Learning** 60 | ![](https://img.shields.io/badge/Mechanism-gray) **Mechanism** 61 | 62 | ### Paper List πŸ“š 63 | 64 | #### 2025 (2025.2-2025.5) πŸ“… 65 | Coming soon! ⏳ 66 | 67 | #### 2024 (2024.11-2025.1) πŸ“… 68 | 69 | - **Spurious Forgetting in Continual Learning of Language Models**, ICLR 2025.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple)![](https://img.shields.io/badge/Continual_Knowledge_Editing-black)![](https://img.shields.io/badge/Continual_Alignment-lightgray) 70 | 71 | - **Knowledge And Capability Transfer Through Large Language Models' Parameters Fusing**, ICLR 2025.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 72 | 73 | - **CollabEdit: Towards Non-destructive Collaborative Knowledge Editing**, ICLR 2025.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 74 | 75 | - **MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code**, ICLR 2025.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 76 | 77 | - **Mix-CPT: A Domain Adaptation Framework via Decoupling Knowledge Learning and Format Alignment**, ICLR 2025.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 78 | 79 | - **Self-Updatable Large Language Models with Parameter Integration**, ICLR 2025.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 80 | 81 | - **Synthetic continued pretraining**, ICLR 2025.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 82 | 83 | - **Last Iterate Convergence of Incremental Methods as a Model of Forgetting**, ICLR 2025. ![](https://img.shields.io/badge/Mechanism-gray) 84 | 85 | - **Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning**, ICLR 2025.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple)![](https://img.shields.io/badge/Mechanism-gray) 86 | 87 | - **Perturbation-Restrained Sequential Model Editing**, ICLR 2025.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 88 | 89 | - **In-Context Editing: Learning Knowledge from Self-Induced Distributions**, ICLR 2025.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 90 | 91 | - **Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective**, AAAI 2025.![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) 92 | 93 | - **CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models**, AAAI 2025.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 94 | 95 | - **Continual Learning Using a Kernel-Based Method Over Foundation Models**, AAAI 2025.![](https://img.shields.io/badge/Continual_Text_Classification-green) 96 | 97 | - **CareBot: A Pioneering Full-Process Open-Source Medical Language Model**, AAAI 2025.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-orange) 98 | 99 | - **Domain-adaptative Continual Learning for Low-resource Tasks: Evaluation on Nepali**, COLING 2025.![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) 100 | 101 | - **In-context Continual Learning Assisted by an External Continual Learner**, COLING 2025.![](https://img.shields.io/badge/Continual_Text_Classification-green) 102 | 103 | - **Continual Learning Using Only Large Language Model Prompting**, COLING 2025.![](https://img.shields.io/badge/Continual_Text_Classification-green) 104 | 105 | - **Rethinking Long Context Generation from the Continual Learning Perspective**, COLING 2025.![](https://img.shields.io/badge/Mechanism-gray) 106 | 107 | - **Dynamic-prototype Contrastive Fine-tuning for Continual Few-shot Relation Extraction with Unseen Relation Detection**, COLING 2025.![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) 108 | 109 | - **Discarding the Crutches: Adaptive Parameter-Efficient Expert Meta-Learning for Continual Semantic Parsing**, COLING 2025. (Semantic Parsing) 110 | 111 | - **TL-CL: Task And Language Incremental Continual Learning**, EMNLP 2024.![](https://img.shields.io/badge/Continual_Machine_Translation-violet) 112 | 113 | - **SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models**, EMNLP 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 114 | 115 | - **Mitigating Catastrophic Forgetting in Language Transfer via Model Merging**, Findings of EMNLP 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 116 | 117 | - **Gradient Localization Improves Lifelong Pretraining of Language Models**, Findings of EMNLP 2024. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-red) 118 | 119 | - **Learn more, but bother less: parameter efficient continual learning**, NIPS 2024.![](https://img.shields.io/badge/Continual_Text_Classification-green) 120 | 121 | - **Continual Learning with Global Alignment**, NIPS 2024.![](https://img.shields.io/badge/Continual_Text_Classification-green) 122 | 123 | - **Continual Audio-Visual Sound Separation**, NIPS 2024. (Audio) 124 | 125 | - **Should We Really Edit Language Models? On the Evaluation of Edited Language Models**, NIPS 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 126 | 127 | - **Continual Learning with Embedding Layer Surgery and Task-wise Beam Search using Whisper**, IEEE Spoken Language Technology Workshop. (Speech) 128 | 129 | - **Chained Tuning Leads to Biased Forgetting**, ICML 2024 Workshop.![](https://img.shields.io/badge/Mechanism-gray) 130 | 131 | - **LOIRE: LifelOng learning on Incremental data via pre-trained language model gRowth Efficiently**, OpenReview Preprint. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 132 | 133 | - **Learning Beyond the Surface: How Far Can Continual Pre-Training with LoRA Enhance LLMs' Domain-Specific Insight Learning?**, Preprint.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 134 | 135 | - **Boosting Tool Use of Large Language Models via Iterative Reinforced Fine-Tuning**, Preprint.![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) 136 | 137 | - **Continual Learning for Encoder-only Language Models via a Discrete Key-Value Bottleneck**, Preprint.![](https://img.shields.io/badge/Continual_Text_Classification-green) 138 | 139 | - **CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search**, Preprint.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 140 | 141 | - **Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training**, Preprint.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 142 | 143 | - **Federated Incremental Named Entity Recognition**, Preprint.![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) 144 | 145 | - **Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty**, Preprint.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 146 | 147 | - **Exploring Forgetting in Large Language Model Pre-Training**, Preprint.![](https://img.shields.io/badge/Mechanism-gray) 148 | 149 | - **Efficient Continual Pre-training of LLMs for Low-resource Languages**, Preprint.![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) 150 | 151 | - **EvoWiki: Evaluating LLMs on Evolving Knowledge**, Preprint.![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-orange) 152 | 153 | #### 2024 (2024.08-2024.10) πŸ“… 154 | 155 | - **WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models**, NIPS 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 156 | 157 | - **Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning**, NIPS 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 158 | 159 | - **D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models**, NIPS 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 160 | 161 | - **TiC-LM: A Multi-Year Benchmark for Continual Pretraining of Language Models**, NIPS 2024 Workshop. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) 162 | 163 | - **Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack**, NIPS 2024 (Database&Benchmark Track)![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) 164 | 165 | - **Does RoBERTa Perform Better than BERT in Continual Learn- ing: An Attention Sink Perspective**, CoLM.![](https://img.shields.io/badge/Mechanism-gray) 166 | 167 | - **LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models**, EMNLP 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 168 | 169 | - **Lifelong Event Detection via Optimal Transport**, EMNLP 2024.![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) 170 | 171 | - **Should We Really Edit Language Models? On the Evaluation of Edited Language Models**, Preprint 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 172 | 173 | - **Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model**, Preprint 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 174 | 175 | - **ToW: Thoughts of Words Improve Reasoning in Large Language Models**, Preprint 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 176 | 177 | - **SeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning**, Preprint 2024.(Speech) 178 | 179 | - **In-context KV-Cache Eviction for LLMs via Attention-Gate**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 180 | 181 | - **In Praise of Stubbornness: The Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 182 | 183 | - **Structure-aware Domain Knowledge Injection for Large Language Models**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 184 | 185 | - **Programming Every Example: Lifting Pre-training Data Quality like Experts at Scale**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 186 | 187 | - **Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance**, Preprint.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 188 | 189 | - **Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 190 | 191 | - **ToEdit: How to Synthesize Text Data to Avoid Model Collapse?**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 192 | 193 | - **Math for AI: On the Generalization of Learning Mathematical Problem Solving**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 194 | 195 | - **EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) 196 | 197 | - **Locate-then-Unlearn: An Effective Method of Multi-Task Continuous Learning for Large Language Models**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 198 | 199 | - **GE-PEFT: Gated Expandable Parameter-Efficient Fine-Tuning for Continual Learning**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Text_Classification-green) 200 | 201 | - **Language Models as Feature Extractors for Accurate Continual Learning**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Text_Classification-green) 202 | 203 | - **Towards Efficient and No Forgetting Domain Continual Pretraining by Mitigating the Stability Gap**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 204 | 205 | - **Data Efficient Continual Learning of Large Language Model**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 206 | 207 | - **NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data**, Preprint Openreview 2024.![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) 208 | 209 | - **Continual Memorization of Factoids in Large Language Models**, Preprint Openreview 2024.![](https://img.shields.io/badge/Mechanism-gray) 210 | 211 | - **Contextual Experience Replay for Continual Learning of Language Agents**, Preprint Openreview 2024.![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) 212 | 213 | - **Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs**, Preprint 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 214 | 215 | - **Is Parameter Collision Hindering Continual Learning in LLMs?**, Preprint 2024.![](https://img.shields.io/badge/Mechanism-gray) 216 | 217 | - **SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture**, Preprint 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 218 | 219 | - **A Closer Look at Machine Unlearning for Large Language Models**, Preprint 2024.![](https://img.shields.io/badge/Mechanism-gray) 220 | 221 | 222 | 223 | - **LeanAgent: Lifelong Learning for Formal Theorem Proving**, Preprint 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 224 | 225 | - **NusaMT-7B: Machine Translation for Low-Resource Indonesian Languages with Large Language Models**, Preprint 2024.![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) 226 | 227 | - **Learning Attentional Mixture of LoRAs for Language Model Continual Learning**, Preprint 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 228 | 229 | - **Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models**, Preprint 2024.![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) 230 | 231 | - **Synthetic continued pretraining**, Preprint 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 232 | 233 | - **Alleviating Hallucinations in Large Language Models with Scepticism Modeling**, Preprint 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) 234 | 235 | - **A Practice of Post-Training on Llama-3 70B with Optimal Selection of Additional Language Mixture Ratio**, Preprint 2024.![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red)![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) 236 | 237 | - **RedWhale: An Adapted Korean LLM Through Efficient Continual Pretraining**, Preprint 2024.![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-red) 238 | 239 | - **PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning**, Preprint 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 240 | 241 | - **Enhance Lifelong Model Editing with Continuous Data-Adapter Association**, Preprint 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 242 | 243 | - **LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin**, ACL 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 244 | 245 | - **TaSL: Task Skill Localization and Consolidation for Language Model Continual Learning**, ACL 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 246 | 247 | - **Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models**, ACL 2024.![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 248 | 249 | - **Continual Dialogue State Tracking via Reason-of-Select Distillation**, Findings of ACL 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 250 | 251 | - **Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System**, Findings of ACL 2024.![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) 252 | 253 | #### 2024 (2024.05-2024.07) πŸ“… 254 | 255 | - **Unlocking Continual Learning Abilities in Language Models**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2406.17245)] [[Code](https://github.com/wenyudu/MIGU)] 256 | 257 | - **Large Language Model Can Continue Evolving From Mistakes**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2404.08707)] 258 | 259 | 260 | - **COPAL: Continual Pruning in Large Language Generative Models**, ICML 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2405.02347)] 261 | 262 | - **Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2404.07143)] 263 | 264 | - **AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2404.08417)] 265 | 266 | - **Efficient Continual Pre-training for Building Domain Specific Large Language Models**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2311.08545)] 267 | 268 | - **Mitigating Catastrophic Forgetting in Language Transfer via Model Merging**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2407.08699v1)] 269 | 270 | - **Reuse, Don’t Retrain: A Recipe for Continued Pretraining of Language Models**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://arxiv.org/pdf/2407.07263v1)] 271 | 272 | - **Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://www.arxiv.org/pdf/2407.02118)] 273 | 274 | - **MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://arxiv.org/pdf/2407.00875)] 275 | 276 | - **Leitner-Guided Memory Replay for Cross-lingual Continual Learning**, NAACL 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://aclanthology.org/2024.naacl-long.432.pdf)] [[Code](https://github.com/meryemmhamdi1/x-continuous-learning/tree/main/humanlearn)] 277 | 278 | - **BAMBINO-LM: (Bilingual-)Human-Inspired Continual Pretraining of BabyLM**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://arxiv.org/pdf/2406.11418)] [[Code](https://github.com/babylm/babylm_data_preprocessing)] 279 | 280 | - **Improving Language Models Trained with Translated Data via Continual Pre-Training and Dictionary Learning Analysis**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://arxiv.org/pdf/2405.14277)] 281 | 282 | - **Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://arxiv.org/pdf/2405.05496)] 283 | 284 | - **ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks**, ACL 2024. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://arxiv.org/pdf/2405.14211)] 285 | 286 | - **XMC-Agent : Dynamic Navigation over Scalable Hierarchical Index for Incremental Extreme Multi-label Classification**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://openreview.net/pdf?id=RQTgBAGNoi)] 287 | 288 | - **Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models**, ACL 2024. ![](https://img.shields.io/badge/Continual_Text_Classification-green) ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://arxiv.org/pdf/2312.07887)] [[Code](https://github.com/zzz47zzz/pretrained-lm-for-incremental-learning)] 289 | 290 | - **Flexible Weight Tuning and Weight Fusion Strategies for Continual Named Entity Recognition**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) 291 | 292 | - **Distilling Causal Effect of Data in Continual Few-shot Relation Learning**, LREC-COLING 2024. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2024.lrec-main.451.pdf)] [[Code](https://github.com/ywh140/CECF)] 293 | 294 | - **Improving Continual Few-shot Relation Extraction through Relational Knowledge Distillation and Prototype Augmentation**, LREC-COLING 2024. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2024.lrec-main.767.pdf)] 295 | 296 | - **Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild**, ACL 2024. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://arxiv.org/pdf/2305.07085)] 297 | 298 | - **An Ensemble-of-Experts Framework for Rehearsal-free Continual Relation Extraction**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) 299 | 300 | - **Continual Few-shot Relation Extraction via Adaptive Gradient Correction and Knowledge Decomposition**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://openreview.net/pdf?id=vOqY1TqNYM)] 301 | 302 | - **Continual Learning with Semi-supervised Contrastive Distillation for Incremental Neural Machine Translation**, ACL 2024. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://openreview.net/pdf?id=iT-OMlYozIs)] 303 | 304 | - **Continual Dialogue State Tracking via Reason-of-Select Distillation**, ACL (Findings) 2024, ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2024.findings-acl.422.pdf)] 305 | 306 | - **Interpretable Catastrophic Forgetting of Large Language Model Fine-tuning via Instruction Vector**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2406.12227v2)] 307 | 308 | - **Learn it or Leave it: Module Composition and Pruning for Continual Learning**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2406.18708)] [[Code](https://github.com/boschresearch/MoCL-Pruning)] 309 | 310 | - **Dirichlet Continual Learning: Tackling Catastrophic Forgetting in NLP**, UAI 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://openreview.net/pdf?id=jve2maFPzf)] 311 | 312 | - **PromptDSI: Prompt-based Rehearsal-free Instance-wise Incremental Learning for Document Retrieval**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2406.12593)] 313 | 314 | - **Revisiting Catastrophic Forgetting in Large Language Model Tuning**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2406.04836)] [[Code](https://github.com/Li-Hyn/LLM_CatastrophicForgetting)] 315 | 316 | - **HFT: Half Fine-Tuning for Large Language Models**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2404.18466)] 317 | 318 | - **Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System**, ACL 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2405.10992)] 319 | 320 | - **SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models**, ACL 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2401.08295)] [[Code](https://github.com/circle-hit/SAPT)] 321 | 322 | - **Self-Evolving GPT: A Lifelong Autonomous Experiential Learner**, ACL 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2407.08937)] 323 | 324 | - **LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin**, ACL 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2312.09979)] [[Code](https://github.com/Ablustrund/LoRAMoE)] 325 | 326 | - **Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuning**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2406.12251)] [[Code](https://github.com/wcyno23/SHLPT)] 327 | 328 | - **Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning**, NAACL(Findings) 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2404.14607)] 329 | 330 | - **Sequential Editing for Lifelong Training of Speech Recognition Models**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://www.arxiv.org/pdf/2406.17935)] 331 | 332 | - **Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://arxiv.org/pdf/2405.03279)] 333 | 334 | - **Detoxifying Large Language Models via Knowledge Editing**, ACL 2024. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://arxiv.org/pdf/2403.14472)] [[Code](https://www.zjukg.org/project/SafeEdit/)] 335 | 336 | - **Model Editing at Scale leads to Gradual and Catastrophic Forgetting**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://arxiv.org/pdf/2401.07453)] 337 | 338 | - **Can We Continually Edit Language Models? On the Knowledge Attenuation in Sequential Model Editing**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) 339 | 340 | - **Incremental Sequential Labeling: A Tale of Two Shifts**, ACL 2024 (Findings). ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://arxiv.org/pdf/2402.10447)] 341 | 342 | - **HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models**, Preprint 2024. ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) [[pdf](https://arxiv.org/pdf/2405.14831)] [[Code](https://github.com/OSU-NLP-Group/HippoRAG)] 343 | 344 | - **Towards Practical Tool Usage for Continually Learning LLMs**, Preprint 2024. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://arxiv.org/pdf/2404.09339)] 345 | 346 | - **LoRA Learns Less and Forgets Less**, Preprint 2024. ![](https://img.shields.io/badge/Mechanism-gray) [[pdf](https://arxiv.org/pdf/2405.09673)] 347 | 348 | 349 | #### 2024 (2024.01-2024.04) πŸ“… 350 | - **CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2402.16767)] [[Code](https://github.com/Sherlock-coder/CorpusBrainPlusPlus)] 351 | 352 | - **Examining Forgetting in Continual Pre-training of Aligned Large Language Models**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2401.03129)] [[Code](https://github.com/lca0503/Llama_tw)] 353 | 354 | - **Rho-1: Not All Tokens Are What You Need**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2404.07965)] [[Code](https://github.com/microsoft/rho)] 355 | 356 | - **SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling**, NAACL 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2312.15166)] 357 | 358 | - **LLaMA Pro: Progressive LLaMA with Block Expansion**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2401.02415)] [[Code](https://github.com/TencentARC/LLaMA-Pro)] 359 | 360 | - **Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learning**, AAAI 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://ojs.aaai.org/index.php/AAAI/article/view/28466/28907)] [[Code](https://github.com/yangbang18/CLFM)] 361 | 362 | - **Simple and Scalable Strategies to Continually Pre-train Large Language Models**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://arxiv.org/pdf/2403.08763)] 363 | 364 | - **Set the Clock: Temporal Alignment of Pretrained Language Models**, ACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) [[pdf](https://arxiv.org/pdf/2402.16797)] [[Code](https://github.com/yizhongw/llm-temporal-alignment)] 365 | 366 | - **HOP to the Next Tasks and Domains for Continual Learning in NLP**, AAAI 2024. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://ojs.aaai.org/index.php/AAAI/article/view/29349/30546 367 | )] 368 | 369 | - **Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://arxiv.org/pdf/2402.12220)] [[Code](https://recherchetts.github.io/bayesian-peft/)] 370 | 371 | - **Rehearsal-Free Modular and Compositional Continual Learning for Language Models**, NAACL 2024. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://arxiv.org/pdf/2404.00790)] 372 | 373 | - **Few-shot Incremental Event Detection**, TALLIP 2024. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://dl.acm.org/doi/pdf/10.1145/3634747)] 374 | 375 | - **Self-generated Replay Memories for Continual Neural Machine Translation**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://arxiv.org/pdf/2403.13130)] [[Code](https://github.com/m-resta/sg-rep)] 376 | 377 | - **F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation**, NAACL 2024. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://arxiv.org/pdf/2404.04846)] [[Code](https://github.com/WJMacro/ContinualMT)] 378 | 379 | - **SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2401.08295)] 380 | 381 | - **InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions**, NAACL 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2403.11435)] 382 | 383 | - **Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2402.18865)] [[Code](https://github.com/which47/LLMCL)] 384 | 385 | - **Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal**, ACL 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2403.01244)] 386 | 387 | - **Scalable Language Model with Generalized Continual Learning**, ICLR 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://openreview.net/pdf?id=mz8owj4DXu)] [[Code](https://github.com/Pbihao/SLM)] 388 | 389 | - **Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language**, NAACL (Findings) 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2404.14607)] 390 | 391 | - **MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://arxiv.org/pdf/2402.11260)] 392 | 393 | - **WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://arxiv.org/pdf/2402.10987)] 394 | 395 | - **CPPO: Continual Learning for Reinforcement Learning with Human Feedback**, ICLR 2024. ![](https://img.shields.io/badge/Continual_Alignment-lightgray) [[pdf](https://openreview.net/pdf?id=86zAUE80pP)] [[Code](https://openi.pcl.ac.cn/Hanlard/CPPO)] 396 | 397 | - **COPR: Continual Human Preference Learning via Optimal Policy Regularization**, Preprint 2024. ![](https://img.shields.io/badge/Continual_Alignment-lightgray) [[pdf](https://arxiv.org/pdf/2402.14228)] [[Code](https://openi.pcl.ac.cn/Hanlard/Offline_alignment_methods_based_on_trlx.git)] 398 | 399 | - **Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection**, ICLR 2024. ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) [[pdf](https://openreview.net/pdf?id=hSyW5go0v8)] [[Code](https://github.com/AkariAsai/self-rag)] 400 | 401 | - **LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error**, Preprint 2024. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://arxiv.org/pdf/2403.04746)] 402 | 403 | - **Towards Practical Tool Usage for Continually Learning LLMs**, Preprint 2024. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://arxiv.org/pdf/2404.09339)] 404 | 405 | - **Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum**, AAAI 2024. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://ojs.aaai.org/index.php/AAAI/article/view/29759/31307)] 406 | 407 | - **Toolllm: Facilitating large language models to master 16000+ real-world apis**, ICLR 2024. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://openreview.net/pdf?id=dHng2O0Jjr)] [[Code](https://github.com/OpenBMB/ToolBench)] 408 | 409 | 410 | #### 2023 πŸ“… 411 | - **Continual Pre-Training of Large Language Models: How to (re)warm your model?**, ICML (Workshop) 2023. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://openreview.net/pdf?id=pg7PUJe0Tl)] 412 | 413 | - **EcomGPT-CT: Continual Pre-training of E-commerce Large Language Models with Semi-structured Data**, Preprint 2023. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2312.15696)] 414 | 415 | - **QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain Search**, KDD 2023. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://dl.acm.org/doi/pdf/10.1145/3580305.3599891)] [[Code](https://github.com/hsaest/QUERT)] 416 | 417 | - **Recyclable Tuning for Continual Pre-training**, ACL (Findings) 2023. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://aclanthology.org/2023.findings-acl.723.pdf)] [[Code](https://github.com/thunlp/RecyclableTuning)] 418 | 419 | - **Large Language Models Encode Clinical Knowledge**, Nature 2023. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2212.13138)] 420 | 421 | - **Continual Learning Under Language Shift**, Preprint 2023. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://arxiv.org/pdf/2311.01200v3)] 422 | 423 | - **Exploring Continual Learning for Code Generation Models**, ACL 2023. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://aclanthology.org/2023.acl-short.68.pdf)] [[Code](https://github.com/amazon-science/codetask-cl-pptf)] 424 | 425 | - **Lifelong Language Pretraining with Distribution-Specialized Experts**, ICML 2023. ![](https://img.shields.io/badge/Continual_Language_Domain_Pretraining-orange) [[pdf](https://openreview.net/pdf?id=Q4QFG5Fe4O)] 426 | 427 | - **Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion**, Preprint 2023. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://arxiv.org/pdf/2305.12270)] 428 | 429 | - **Rehearsal-free Continual Language Learning via Efficient Parameter Isolation**, ACL 2023. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2023.acl-long.612.pdf)] [[Code](https://github.com/Dicer-Zz/EPI)] 430 | 431 | - **Class-Incremental Learning based on Label Generation**, ACL 2023. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2023.acl-short.109.pdf)] [[Code](https://github.com/shaoyijia/VAG)] 432 | 433 | - **Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning**, ACL (Findings) 2023. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2023.findings-acl.48.pdf)] 434 | 435 | - **InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective**, EMNLP 2023. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2023.findings-emnlp.969.pdf)] [[Code](https://github.com/Yifan-Song793/InfoCL)] 436 | 437 | - **ConPET: Continual Parameter-Efficient Tuning for Large Language Models**, Preprint 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://arxiv.org/pdf/2309.14763)] [[Code](https://github.com/Raincleared-Song/ConPET)] 438 | 439 | - **A Neural Span-Based Continual Named Entity Recognition Model**, AAAI 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://ojs.aaai.org/index.php/AAAI/article/view/26638/26410)] [[Code](https://github.com/Qznan/SpanK)] 440 | 441 | - **Learning β€œO” Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER**, ACL 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2023.acl-long.328.pdf)] [[Code](https://github.com/rtmaww/O_CILNER)] 442 | 443 | - **Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction**, ACL (Findings) 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2023.findings-acl.141.pdf)] [[Code](https://github.com/VT-NLP/ICE)] 444 | 445 | - **ProtoNER: Few Shot Incremental Learning for Named Entity Recognition Using Prototypical Networks**, BPM 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://arxiv.org/pdf/2310.02372)] 446 | 447 | - **Task Relation Distillation and Prototypical Pseudo Label for Incremental Named Entity Recognition**, CIKM 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://arxiv.org/pdf/2308.08793)] [[Code](https://github.com/BladeDancer957/INER_RDP)] 448 | 449 | - **Continual Named Entity Recognition without Catastrophic Forgetting**, CIKM 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2023.emnlp-main.509.pdf)] [[Code](https://github.com/BladeDancer957/CPFD)] 450 | 451 | - **SKD-NER:Continual Named Entity Recognition via Span-based Knowledge Distillation with Reinforcement Learning**, EMNLP 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2023.emnlp-main.413.pdf)] 452 | 453 | - **Novel Slot Detection With an Incremental Setting**, EMNLP (Findings) 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2023.findings-emnlp.53.pdf)] [[Code](https://github.com/cs-liangchen-work/NovelIE)] 454 | 455 | - **Incremental event detection via an improved knowledge distillation based model**, Neurocomputing 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://pdf.sciencedirectassets.com/271597/1-s2.0-S0925231223X00289/1-s2.0-S0925231223006422/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEIb%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJGMEQCIEK1bj6Ly8ruhs9a2Zb2c5uJ49vKzMhJlcbiIbJJpcIZAiAFc1xzJtgK%2FpO%2Fkbz2KJxccOqTyVgGBW7HZ%2BEDgnGcVyq8BQjv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAUaDDA1OTAwMzU0Njg2NSIMWcVdnvyvyvCOf613KpAFTU%2FIhOI61FrBYfhW7wvTWc9Iia5pvrE5PmD2tHyfFtCTvkgVYnDSXfnIsGK%2FZMF0jZARLZ5NlXyPfR1%2B7HZiZ1vaDvadAfG0sLKtY%2ByplTKXA0FzrfRBcj0oG6%2F90Kf7bKQbj6eN3JexNRUIHneHOWcrlFHbZYuIroDvXNKa9ieEBj0pDBMFckBbvc%2BwuNzsZbipC5581zjP4joiiPf1oxw1VaClYKxGzRsyZGIwN5UxNX3kWxuAccBS%2FSDRcA2XpIg8wPEVT16FFSYmrK2tZ2tk2OjKGIu66Md9nUS4YV3BK5SCVGEfTug18d5Etwn%2FpTouhyVBcjUDF0ZR6bKREbFJyQQa6fC2W%2B62JmyYLxu2QSmqI%2F8SBV0a5C7KaYxiqHl0Ot4Yy%2BsIRe8zFkyw1EzLC%2BZEMED5KLyDztFn15mYnLdUu9sIrUBf6cXvmSzhDi7%2FSou7Teww4YFQwdQ6bnfyRZgeF%2B5BKli9d9WUWZmQE5Hp%2FAcQxGtdFCMiOkiB31uIbcbaBIu4%2Fg1WzNj0dtyu%2Fdrxahce2QdqyXpV5fLMepPzvNR1n6TzAX6K0otDDa3tX%2Fd%2Bzwe1TRX7DpkDDuC5yFWnZOAT7lGssLMH9aHZf0SZSv1JPxHk41ZY9MXiXXFqawtL1%2BcIoxoKVN3wJF6BGfcp3Yz98jv13D316X841Haitq85oBuwXCdb9QH6nKDZFn2NhinCVC5UhiRGTJItPxKdO9RKAsdrjX3lhJHiMT%2FOhUWAf%2BYvPdV1ukxUfgiA0ZHcqQ8AL4Rg1%2B9GPZZvTjgr6HbVRzsGUgNbRE38XJy0RAfuBPfOQcyyC2q6Y%2BdXj9Qxhcj7cErgw2Y15s4Gbu9O3RT3YDCmIOGCU4YwweeisgY6sgGyC5MQGJB5takEqm5cLtpQC2HodtOM70AANOdIQSGE%2Fl1d9JswbIXx2nyGlFFmxm6426QLCkdHe5PHWAhpLkmKWyN6X8krQ3H5617W%2FUe1cTM0SLHkKi6SoDe0QfpBiTNpKPQn%2BuczGYP3YeFGybQEaet5nWtK0YOd41MCQO2wWVHK1CuLQwJ9ZYlef4Aq9wtDXVpCSRaGfc1TmmolhHDOfC7YZY7dQNtHVz8GPYGHLjBp&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240518T150903Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&sX-Amz-Credential=ASIAQ3PHCVTYRJJBNUDJ%2F20240518%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=958666ecc95b23232cf77236b4e51d1057271ab605b99aaf87ac92cfc0f904a7&hash=9b3545292353e6576b86cbb5b53f1cf0563d2f2676821cdfd397782aa16669d9&host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&pii=S0925231223006422&tid=spdf-130a1314-9dc9-458d-8716-9d499548602c&sid=3682f588522cd0438f8ac403cedfc1eb08f0gxrqa&type=client&tsoh=d3d3LnNjaWVuY2VkaXJlY3QuY29t&ua=0d095d565b0d0355&rr=885cc41e4dff84b5&cc=cn)] 456 | 457 | - **Decomposing Logits Distillation for Incremental Named Entity Recognition**, SIGIR 2023. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://dl.acm.org/doi/pdf/10.1145/3539618.3591970)] 458 | 459 | - **Consistent Prototype Learning for Few-Shot Continual Relation Extraction**, ACL 2023. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2023.acl-long.409.pdf)] [[Code](https://github.com/XiudiChen/ConPL)] 460 | 461 | - **Enhancing Continual Relation Extraction via Classifier Decomposition**, ACL (Findings) 2023. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2023.findings-acl.638.pdf)] [[Code](https://github.com/hemingkx/CDec)] 462 | 463 | - **Improving Continual Relation Extraction by Distinguishing Analogous Semantics**, ACL 2023. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2023.acl-long.65.pdf)] [[Code](https://github.com/nju-websoft/CEAR)] 464 | 465 | - **Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction**, ACL (Findings) 2023. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2023.findings-acl.804.pdf)] [[Code](https://github.com/nju-websoft/SCKD)] 466 | 467 | - **ICA-Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification**, EACL (Findings) 2023. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2023.findings-eacl.171.pdf)] 468 | 469 | - **Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction**, ACL (Findings) 2023. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2023.findings-acl.804.pdf)] [[Code](https://github.com/nju-websoft/SCKD)] 470 | 471 | - **Continual Knowledge Distillation for Neural Machine Translation**, ACL 2023. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2023.acl-long.443.pdf)] [[Code](https://github.com/THUNLP-MT/CKD)] 472 | 473 | - **Knowledge Transfer in Incremental Learning for Multilingual Neural Machine Translation**, ACL 2023. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2023.acl-long.852.pdf)] [[Code](https://github.com/THUNLP-MT/ktnmt)] 474 | 475 | - **Continual Learning for Multilingual Neural Machine Translation via Dual Importance-based Model Division**, EMNLP 2023. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2023.emnlp-main.736.pdf)] [[Code](https://github.com/raburabu91/BVP4CL)] 476 | 477 | - **Prompts Can Play Lottery Tickets Well: Achieving Lifelong Information Extraction via Lottery Prompt Tuning**, ACL 2023. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2023.acl-long.16.pdf)] [[Code](https://github.com/jokieleung/Lottery_Prompt)] 478 | 479 | - **Large-scale Lifelong Learning of In-context Instructions and How to Tackle It**, ACL 2023. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2023.acl-long.703.pdf)] 480 | 481 | - **Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning**, EACL 2023. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2023.eacl-main.65.pdf)] [[Code](https://github.com/arumaekawa/GR-HMI)] 482 | 483 | - **Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation**, EMNLP 2023. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2023.emnlp-main.414.pdf)] 484 | 485 | - **Orthogonal Subspace Learning for Language Model Continual Learning**, EMNLP (Findings) 2023. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2023.findings-emnlp.715.pdf)] [[Code](https://github.com/cmnfriend/O-LoRA)] 486 | 487 | - **Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks**, EMNLP (Findings) 2023. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2023.findings-emnlp.1008.pdf)] 488 | 489 | - **Progressive Prompts: Continual Learning for Language Models**, ICLR 2023. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://openreview.net/pdf?id=UJTgQBc91_)] [[Code](https://github.com/arazd/ProgressivePrompts)] 490 | 491 | - **Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models**, EACL 2023. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://aclanthology.org/2023.eacl-main.199.pdf)] [[Code](https://github.com/peterbhase/SLAG-Belief-Updating)] 492 | 493 | - **Aging with GRACE: Lifelong Model Editing with Key-Value Adaptors**, ICLR 2023. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://openreview.net/pdf?id=ngCT1EelZk)] 494 | 495 | - **Transformer-Patcher: One Mistake worth One Neuron**, ICLR 2023. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://openreview.net/pdf?id=4oYUGeGBPm)] [[Code](https://github.com/ZeroYuHuang/Transformer-Patcher)] 496 | 497 | - **Mitigating the Alignment Tax of RLHF**, Preprint 2023. ![](https://img.shields.io/badge/Continual_Alignment-lightgray) [[pdf](https://arxiv.org/pdf/2309.06256)] 498 | 499 | - **Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models**, EMNLP 2023. ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) [[pdf](https://aclanthology.org/2023.emnlp-main.63.pdf)] [[Code](https://github.com/gankim/tree-of-clarifications)] 500 | 501 | - **Active Retrieval Augmented Generation**, EMNLP 2023. ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) [[pdf](https://aclanthology.org/2023.emnlp-main.495.pdf)] [[Code](https://github.com/jzbjyb/FLARE)] 502 | 503 | - **Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions**, ACL 2023. ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) [[pdf](https://aclanthology.org/2023.acl-long.557.pdf)] [[Code](https://github.com/stonybrooknlp/ircot)] 504 | 505 | - **Toolalpaca: Generalized tool learning for language models with 3000 simulated cases**, Preprint 2023. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://arxiv.org/pdf/2306.05301)] [[Code](https://github.com/tangqiaoyu/ToolAlpaca)] 506 | 507 | - **Gorilla: Large language model connected with massive apis**, Preprint 2023. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://arxiv.org/pdf/2305.15334)] [[Code](https://github.com/ShishirPatil/gorilla)] 508 | 509 | - **Chameleon: Plug-and-play compositional reasoning with large language models**, NIPS 2023. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://openreview.net/pdf?id=HtqnVSCj3q)] [[Code](https://github.com/lupantech/chameleon-llm)] 510 | 511 | - **Toolformer: Language Models Can Teach Themselves to Use Tools**, NIPS 2023. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://openreview.net/pdf?id=Yacmpz84TH)] 512 | 513 | - **GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction**, NIPS 2023. ![](https://img.shields.io/badge/Tool_Based_Lifelong_Learning-khaki) [[pdf](https://openreview.net/pdf?id=cwjh8lqmOL)] [[Code](https://github.com/AILab-CVC/GPT4Tools)] 514 | 515 | 516 | #### 2022 πŸ“… 517 | - **Continual Pre-Training Mitigates Forgetting in Language and Vision**, Preprint 2022. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://arxiv.org/pdf/2205.09357)] [[Code](https://github.com/AndreaCossu/continual-pretraining-nlp-vision)] 518 | 519 | - **ELLE: Efficient Lifelong Pre-training for Emerging Data**, ACL (Findings) 2022. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://aclanthology.org/2022.findings-acl.220.pdf)] [[Code](https://github.com/thunlp/ELLE)] 520 | 521 | - **Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora**, ACL (Workshop) 2022. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://aclanthology.org/2022.bigscience-1.1.pdf)] 522 | 523 | - **bert2BERT: Towards Reusable Pretrained Language Models**, ACL 2022. ![](https://img.shields.io/badge/Continual_Vertical_Domain_Pretraining-red) [[pdf](https://aclanthology.org/2022.acl-long.151.pdf)] [[Code](https://github.com/huawei-noah/Pretrained-Language-Model)] 524 | 525 | - **Time Waits for No One! Analysis and Challenges of Temporal Misalignment**, NAACL 2022. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) [[pdf](https://aclanthology.org/2022.naacl-main.435.pdf)] [[Code](https://github.com/Kel-Lu/time-waits-for-no-one)] 526 | 527 | - **TimeLMs: Diachronic Language Models from Twitter**, ACL 2022. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) [[pdf](https://aclanthology.org/2022.acl-demo.25.pdf)] [[Code](https://github.com/cardiffnlp/timelms)] 528 | 529 | - **Continual Few-shot Intent Detection**, COLING 2022. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2022.coling-1.26.pdf)] 530 | 531 | - **Incremental Intent Detection for Medical Domain with Contrast Replay Networks**, ACL (Findings) 2022. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2022.findings-acl.280.pdf)] 532 | 533 | - **Continual training of language models for few-shot learning**, EMNLP 2022. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2022.emnlp-main.695.pdf)] 534 | 535 | - **Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System**, NAACL 2022. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2022.naacl-industry.35.pdf)] 536 | 537 | - **Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection**, NAACL (Findings) 2022. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2022.findings-naacl.84.pdf)] 538 | 539 | - **Memory Efficient Continual Learning with Transformers**, NIPS 2022. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://openreview.net/pdf?id=U07d1Y-x2E)] 540 | 541 | - **Few-Shot Class-Incremental Learning for Named Entity Recognition**, ACL 2022. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2022.acl-long.43.pdf)] 542 | 543 | - **Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples**, ACL (Findings) 2022. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2022.findings-acl.179.pdf)] 544 | 545 | - **Incremental Prompting: Episodic Memory Prompt for Lifelong Event Dectection**, COLING 2022. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2022.coling-1.189.pdf)] [[Code](https://github.com/VT-NLP/Incremental_Prompting)] 546 | 547 | - **Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition**, EMNLP 2022. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2022.emnlp-main.236.pdf)] [[Code](https://github.com/zzz47zzz/CFNER)] 548 | 549 | - **BNU: A Balance-Normalization-Uncertainty Model for Incremental Event Detection**, ICASSP 2022. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9747708)] 550 | 551 | - **Similarity-Driven Adaptive Prototypical Network for Class-incremental Few-shot Named Entity Recognition**, ICTAI 2022. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10097986)] 552 | 553 | - **HEFT: A History-Enhanced Feature Transfer framework for incremental event detection**, KBS 2022. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://pdf.sciencedirectassets.com/271505/1-s2.0-S0950705122X00166/1-s2.0-S0950705122008061/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEIb%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIDGej%2BZ9m2J2ytK5w36KWKcEv3gThuaWIHufatpHidkTAiEAjJYEcHYhW01TAkVvFeUvqUbfrBdHwQcYMnxxmjet93YquwUI7%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAFGgwwNTkwMDM1NDY4NjUiDKzZZZZI3SMAHjMbuyqPBaAhyjLItjCl3zFmft%2BxDfWV2MfRrhy5%2Bdf5DpBoJjDrC4IrsKr5iocT0pD31pIiLMMSElG7KE84hTmE8dgkX5%2FRloLcvTuKCjuVs6fVJvRCspsveqr7upGzXl0DgKrMbCsGO%2BegT%2BDP7YcoHveQbZIryNAucTz7YJAU5NzAaF9ZbMV8UkiGfiVeo4PzK7hfliJuIYhpmCOE9PQsmuXlOlJIojXndgiJwW0x53Zk6zji9oxbbtHkCtDtG4fmK%2BAcjjarclM5Ih9iSHSU32N1Ez8FTGLt98Ta6T4AdIUjZynMR%2FVQCg4s9w%2BZup%2FUma3vsKvCdUGTNLAhugf6k9Y2p5%2FkEpyzxAuVAE5cbkthVbfZwAbpQZZlausIsq8zZoj%2F9A9liO8h4L4s%2BJ5WVA5sH3WTMAlVwFoW6nNIGSj3tbhWWudVvkrBcqTW5DnG%2BTSKH4Qns%2FEweyusiLmzx0eTQB9JjVdwgDs5iFO2KvQdmn%2F2NfLjSKJY1mO0TN%2BLn1f5G%2FyCBmthNJyNz0dyrUzkt5u%2Fvp6SBXXbwezklK%2FZJiAgKK7m4P4ocS%2B5yC%2B1ueXS2REcznQRFRzTIhU6kcT4uDt92wqIIaN23jQ16QVgn8U27%2Fn1kzEFqqh5FFly3r6zYOFgNtz8OuX9vdCif3pIwuZb9TDpCdQ6mIC7z3pWHzFwvHPBhmzHmI7WMSn%2FROlEXid2%2FpzFSe4laBDIgQv0yDa8BxlKHlZxt3vHi51z%2FXgZRBttoRsnU4H2cBPhUBoxDvcgsEntJF%2Br4Lm2DTFOm2mwKiND7tBcijwxErP9caOOZ6ZdLSeU9NYXYdrC8yMhUIFEtG9ysZaFLBp6AjQeCNIJsXrXrbBmmh3t8xukPLUwrumisgY6sQFh7rJYe3CCejqSjvILn5KdJ%2BE%2BiPN1D7zZ%2Fun1nTwKoH3U%2FktdVr%2FOmjv22mOdUduSvEv69pq97JwEYOmlIp42GlobQ9kzOriOiJ727M92UfjMhSPrwTcSmvLeTb54gLhmcLuiS0WZBdUfx%2F2XEYuahDYuLrcbCTH2m4InSytNbOeAWWIlTAnosDjEsHd2JlSk1V70ej13StBo5ZXYjfNllXbd2R%2Fw1kMOoYZTAK9QI9E%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240518T150930Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIAQ3PHCVTY5532AY56%2F20240518%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=0f8e27f71a9a02306eee68911d2f0c2052ddbb046c6b91e476011c97eb80ec65&hash=e2913561864f523ed8763a508a574ea73fa2022b9b6490a137d91cdc987b830a&host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&pii=S0950705122008061&tid=spdf-4425fead-8428-44a4-85ca-dc5f5464a640&sid=3682f588522cd0438f8ac403cedfc1eb08f0gxrqa&type=client&tsoh=d3d3LnNjaWVuY2VkaXJlY3QuY29t&ua=0d095d565b0d0355&rr=885cc4c5cb9284b5&cc=cn)] 554 | 555 | - **Consistent Representation Learning for Continual Relation Extraction**, ACL (Findings) 2022. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2022.findings-acl.268.pdf)] [[Code](https://github.com/thuiar/CRL)] 556 | 557 | - **Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation**, ACL 2022. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2022.acl-long.198.pdf)] [[Code](https://github.com/qcwthu/Continual_Fewshot_Relation_Learning)] 558 | 559 | - **Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach**, Preprint 2022. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2022.acl-long.198.pdf)] 560 | 561 | - **Improving continual relation extraction through prototypical contrastive learning**, COLING 2022. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2022.coling-1.163.pdf)] [[Code](https://github.com/PaperDiscovery/CRECL)] 562 | 563 | - **Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation**, EMNLP 2022. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2022.emnlp-main.420.pdf)] [[Code](https://github.com/Wangpeiyi9979/ACA)] 564 | 565 | - **Prompt-based prototypical framework for continual relation extraction**, TASLP 2022. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://dl.acm.org/doi/pdf/10.1109/TASLP.2022.3199655)] 566 | 567 | - **Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation**, ACL 2022. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2022.acl-long.143.pdf)] [[Code](https://github.com/ictnlp/COKD)] 568 | 569 | - **CLLE: A Benchmark for Continual Language Learning Evaluation in Multilingual Machine Translation**, EMNLP (Findings) 2022. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2022.findings-emnlp.30.pdf)] [[Code](https://github.com/HITSZ-HLT/CLLE)] 570 | 571 | - **Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions**, EMNLP 2022. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2022.emnlp-main.111.pdf)] [[Code](https://github.com/ictnlp/LFR-NMT)] 572 | 573 | - **Entropy-Based Vocabulary Substitution for Incremental Learning in Multilingual Neural Machine Translation**, EMNLP 2022. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2022.emnlp-main.720/)] [[Code](https://github.com/koukaiu/evs)] 574 | 575 | - **Continual Sequence Generation with Adaptive Compositional Modules**, ACL 2022. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2022.acl-long.255.pdf)] [[Code](https://github.com/GT-SALT/Adaptive-Compositional-Modules)] 576 | 577 | - **ConTinTin: Continual Learning from Task Instructions**, ACL 2022. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2022.acl-long.218.pdf)] 578 | 579 | - **Continual Prompt Tuning for Dialog State Tracking**, ACL 2022. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2022.acl-long.80.pdf)] [[Code](https://github.com/thu-coai/cpt4dst)] 580 | 581 | - **Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue**, EMNLP 2022. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2022.emnlp-main.766.pdf)] [[Code](https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/pcll)] 582 | 583 | - **Fine-tuned Language Models are Continual Learners**, EMNLP 2022. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2022.emnlp-main.410.pdf)] [[Code](https://github.com/ThomasScialom/T0_continual_learning)] 584 | 585 | - **LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5**, ICLR 2022. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://openreview.net/pdf?id=HCRVf71PMF)] [[Code](https://github.com/qcwthu/Lifelong-Fewshot-Language-Learning)] 586 | 587 | - **Plug-and-Play Adaptation for Continuously-updated QA**, ACL (Findings) 2022. ![](https://img.shields.io/badge/Continual_Knowledge_Editing-black) [[pdf](https://aclanthology.org/2022.findings-acl.37.pdf)] [[Code](https://github.com/wookjeHan/Continual-Plug-and-Adapt-for-CuQA/)] 588 | 589 | 590 | #### 2021 πŸ“… 591 | - **Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media**, EMNLP (Findings) 2021. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) [[pdf](https://aclanthology.org/2021.findings-emnlp.206.pdf)] [[Code](https://github.com/paul-rottger/temporal-adaptation)] 592 | 593 | - **ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning**, EMNLP 2021. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) [[pdf](https://aclanthology.org/2021.emnlp-main.436.pdf)] [[Code](https://github.com/PlusLabNLP/ECONET)] 594 | 595 | - **Mind the Gap: Assessing Temporal Generalization in Neural Language Models**, NIPS 2021. ![](https://img.shields.io/badge/Continual_Temporal_Domain_Pretraining-yellow) [[pdf](https://openreview.net/pdf?id=73OmmrCfSyy)] [[Code](https://github.com/google-deepmind/deepmind-research/tree/master/pitfalls_static_language_models)] 596 | 597 | - **Learning to Solve NLP Tasks in an Incremental Number of Languages**, ACL 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.acl-short.106.pdf)] 598 | 599 | - **Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks**, NAACL 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.naacl-main.378.pdf)] [[Code](https://github.com/ZixuanKe/PyContinual)] 600 | 601 | - **CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks**, EMNLP 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.emnlp-main.550.pdf)] [[Code](https://github.com/ZixuanKe/PyContinual)] 602 | 603 | - **Continual Learning for Text Classification with Information Disentanglement Based Regularization**, NAACL 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.naacl-main.218.pdf)] [[Code](https://github.com/GT-SALT/IDBR)] 604 | 605 | - **Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding**, NAACL 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.naacl-main.212.pdf)] [[Code](https://github.com/tinghua-code/CCFI)] 606 | 607 | - **Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System**, NAACL 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.naacl-main.106.pdf)] [[Code](https://github.com/congyingxia/IncrementalFSTC)] 608 | 609 | - **Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning**, NIPS 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://proceedings.neurips.cc/paper_files/paper/2021/file/bcd0049c35799cdf57d06eaf2eb3cff6-Paper.pdf)] [[Code](https://github.com/ZixuanKe/PyContinual)] 610 | 611 | - **Iterative Network Pruning with Uncertainty Regularization for Lifelong Sentiment Classification**, SIGIR 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://dl.acm.org/doi/pdf/10.1145/3404835.3462902)] [[Code](https://github.com/siat-nlp/IPRLS)] 612 | 613 | - **Lifelong Intent Detection via Multi-Strategy Rebalancing**, Preprint 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://arxiv.org/pdf/2106.11197)] 614 | 615 | - **Lifelong Knowledge-Enriched Social Event Representation Learning**, EACL 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.eacl-main.317.pdf)] [[Code](https://pralav.github.io/lifelong_eventrep/?c=10)] 616 | 617 | - **Lifelong Learning of Hate Speech Classification on Social Media**, NAACL 2021. ![](https://img.shields.io/badge/Continual_Text_Classification-green) [[pdf](https://aclanthology.org/2021.naacl-main.183.pdf)] 618 | 619 | - **Continual Learning for Named Entity Recognition**, AAAI 2021. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://ojs.aaai.org/index.php/AAAI/article/view/17600/17407)] 620 | 621 | - **Lifelong Event Detection with Knowledge Transfer**, EMNLP 2021. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2021.emnlp-main.428.pdf)] [[Code](https://github.com/Perfec-Yu/Lifelong-ED)] 622 | 623 | - **Curriculum-meta learning for order-robust continual relation extraction**, AAAI 2021. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://ojs.aaai.org/index.php/AAAI/article/view/17241/17048)] [[Code](https://github.com/wutong8023/AAAI-CML)] 624 | 625 | - **Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction**, ACL 2021. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2021.acl-long.20.pdf)] [[Code](https://github.com/fd2014cl/RP-CRE)] 626 | 627 | - **Continual learning in multilingual nmt via language-specific embeddings**, WMT 2021. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2021.wmt-1.62/)] 628 | 629 | - **Continual Learning for Neural Machine Translation**, NAACL 2021. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2021.naacl-main.310/)] [[Code](https://github.com/caoy1996/CLNMT)] 630 | 631 | - **Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution**, NAACL 2021. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2021.naacl-main.93.pdf)] 632 | 633 | - **Rational LAMOL: A rationale-based lifelong learning framework**, ACL 2021. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2021.acl-long.229.pdf)] [[Code](https://github.com/kanwatchara-k/r_lamol)] 634 | 635 | - **Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking**, ACL 2021. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2021.acl-short.66.pdf)] [[Code](https://github.com/siat-nlp/TPEM)] 636 | 637 | - **Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning**, EMNLP (Findings) 2021. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2021.findings-emnlp.62.pdf)] [[Code](https://github.com/INK-USC/CLIF)] 638 | 639 | - **Continual Learning in Task-Oriented Dialogue Systems**, EMNLP 2021. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2021.emnlp-main.590.pdf)] [[Code](https://github.com/andreamad8/ToDCL)] 640 | 641 | 642 | #### 2020 πŸ“… 643 | - **Incremental Event Detection via Knowledge Consolidation Networks**, EMNLP 2020. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/2020.emnlp-main.52.pdf)] [[Code](https://github.com/CPF-NLPR/IncrementalED)] 644 | 645 | - **Continual Relation Learning via Episodic Memory Activation and Reconsolidation**, ACL 2020. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/2020.acl-main.573.pdf)] [[Code](https://github.com/thunlp/ContinualRE)] 646 | 647 | - **Findings of the First Shared Task on Lifelong Learning Machine Translation**, WMT 2020. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/2020.wmt-1.2.pdf)] 648 | 649 | - **Distill and Replay for Continual Language Learning**, COLING 2020. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2020.coling-main.318.pdf)] 650 | 651 | - **Continual Learning for Natural Language Generation in Task-oriented Dialog Systems**, EMNLP (Findings) 2020. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/2020.findings-emnlp.310.pdf)] [[Code](https://github.com/MiFei/Continual-Learning-for-NLG)] 652 | 653 | - **LAMOL: LAnguage MOdeling for Lifelong Language Learning**, ICLR 2020. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://openreview.net/pdf?id=Skgxcn4YDS)] [[Code](https://github.com/jojotenya/LAMOL)] 654 | 655 | - **Dense Passage Retrieval for Open-Domain Question Answering**, EMNLP 2020. ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) [[pdf](https://aclanthology.org/2020.emnlp-main.550.pdf)] [[Code](https://github.com/facebookresearch/DPR)] 656 | 657 | - **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**, NIPS 2020. ![](https://img.shields.io/badge/Retrieval_Based_Lifelong_Learning-beige) [[pdf](https://proceedings.neurips.cc/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf)] [[Code](https://github.com/huggingface/transformers)] 658 | 659 | 660 | #### 2019 πŸ“… 661 | - **A Progressive Model to Enable Continual Learning for Semantic Slot Filling**, EMNLP 2019. ![](https://img.shields.io/badge/Continual_Named_Entity_Recognition-brightgreen) [[pdf](https://aclanthology.org/D19-1126.pdf)] 662 | 663 | - **Meta-Learning Improves Lifelong Relation Extraction**, RepL4NLP 2019. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/W19-4326.pdf)] 664 | 665 | - **Sentence Embedding Alignment for Lifelong Relation Extraction**, NAACL 2019. ![](https://img.shields.io/badge/Continual_Relation_Extraction-blue) [[pdf](https://aclanthology.org/N19-1086.pdf)] [[Code](https://github.com/hongwang600/Lifelong_Relation_Detection)] 666 | 667 | - **From Bilingual to Multilingual Neural Machine Translation by Incremental Training**, JASIST 2019. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/P19-2033.pdf)] 668 | 669 | - **Incremental Learning from Scratch for Task-Oriented Dialogue Systems**, ACL 2019. ![](https://img.shields.io/badge/Continual_Instruction_Tuning-purple) [[pdf](https://aclanthology.org/P19-1361.pdf)] [[Code](https://github.com/Leechikara/Incremental-Dialogue-System)] 670 | 671 | 672 | #### 2018 πŸ“… 673 | - **Regularized Training Objective for Continued Training for Domain Adaptation in Neural Machine Translation**, NGT 2018. ![](https://img.shields.io/badge/Continual_Machine_Translation-violet) [[pdf](https://aclanthology.org/W18-2705.pdf)] [[Code](https://github.com/khayrallah/OpenNMT-py-reg)] 674 | --------------------------------------------------------------------------------