├── fig ├── active_learning_cycle.jpg └── an_illustrative_AL_example.jpg ├── LICENSE └── README.md /fig/active_learning_cycle.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/baifanxxx/awesome-active-learning/HEAD/fig/active_learning_cycle.jpg -------------------------------------------------------------------------------- /fig/an_illustrative_AL_example.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/baifanxxx/awesome-active-learning/HEAD/fig/an_illustrative_AL_example.jpg -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Creative Commons Legal Code 2 | 3 | CC0 1.0 Universal 4 | 5 | CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE 6 | LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN 7 | ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS 8 | INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES 9 | REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS 10 | PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM 11 | THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED 12 | HEREUNDER. 13 | 14 | Statement of Purpose 15 | 16 | The laws of most jurisdictions throughout the world automatically confer 17 | exclusive Copyright and Related Rights (defined below) upon the creator 18 | and subsequent owner(s) (each and all, an "owner") of an original work of 19 | authorship and/or a database (each, a "Work"). 20 | 21 | Certain owners wish to permanently relinquish those rights to a Work for 22 | the purpose of contributing to a commons of creative, cultural and 23 | scientific works ("Commons") that the public can reliably and without fear 24 | of later claims of infringement build upon, modify, incorporate in other 25 | works, reuse and redistribute as freely as possible in any form whatsoever 26 | and for any purposes, including without limitation commercial purposes. 27 | These owners may contribute to the Commons to promote the ideal of a free 28 | culture and the further production of creative, cultural and scientific 29 | works, or to gain reputation or greater distribution for their Work in 30 | part through the use and efforts of others. 31 | 32 | For these and/or other purposes and motivations, and without any 33 | expectation of additional consideration or compensation, the person 34 | associating CC0 with a Work (the "Affirmer"), to the extent that he or she 35 | is an owner of Copyright and Related Rights in the Work, voluntarily 36 | elects to apply CC0 to the Work and publicly distribute the Work under its 37 | terms, with knowledge of his or her Copyright and Related Rights in the 38 | Work and the meaning and intended legal effect of CC0 on those rights. 39 | 40 | 1. Copyright and Related Rights. A Work made available under CC0 may be 41 | protected by copyright and related or neighboring rights ("Copyright and 42 | Related Rights"). Copyright and Related Rights include, but are not 43 | limited to, the following: 44 | 45 | i. the right to reproduce, adapt, distribute, perform, display, 46 | communicate, and translate a Work; 47 | ii. moral rights retained by the original author(s) and/or performer(s); 48 | iii. publicity and privacy rights pertaining to a person's image or 49 | likeness depicted in a Work; 50 | iv. rights protecting against unfair competition in regards to a Work, 51 | subject to the limitations in paragraph 4(a), below; 52 | v. rights protecting the extraction, dissemination, use and reuse of data 53 | in a Work; 54 | vi. database rights (such as those arising under Directive 96/9/EC of the 55 | European Parliament and of the Council of 11 March 1996 on the legal 56 | protection of databases, and under any national implementation 57 | thereof, including any amended or successor version of such 58 | directive); and 59 | vii. other similar, equivalent or corresponding rights throughout the 60 | world based on applicable law or treaty, and any national 61 | implementations thereof. 62 | 63 | 2. Waiver. To the greatest extent permitted by, but not in contravention 64 | of, applicable law, Affirmer hereby overtly, fully, permanently, 65 | irrevocably and unconditionally waives, abandons, and surrenders all of 66 | Affirmer's Copyright and Related Rights and associated claims and causes 67 | of action, whether now known or unknown (including existing as well as 68 | future claims and causes of action), in the Work (i) in all territories 69 | worldwide, (ii) for the maximum duration provided by applicable law or 70 | treaty (including future time extensions), (iii) in any current or future 71 | medium and for any number of copies, and (iv) for any purpose whatsoever, 72 | including without limitation commercial, advertising or promotional 73 | purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each 74 | member of the public at large and to the detriment of Affirmer's heirs and 75 | successors, fully intending that such Waiver shall not be subject to 76 | revocation, rescission, cancellation, termination, or any other legal or 77 | equitable action to disrupt the quiet enjoyment of the Work by the public 78 | as contemplated by Affirmer's express Statement of Purpose. 79 | 80 | 3. Public License Fallback. Should any part of the Waiver for any reason 81 | be judged legally invalid or ineffective under applicable law, then the 82 | Waiver shall be preserved to the maximum extent permitted taking into 83 | account Affirmer's express Statement of Purpose. In addition, to the 84 | extent the Waiver is so judged Affirmer hereby grants to each affected 85 | person a royalty-free, non transferable, non sublicensable, non exclusive, 86 | irrevocable and unconditional license to exercise Affirmer's Copyright and 87 | Related Rights in the Work (i) in all territories worldwide, (ii) for the 88 | maximum duration provided by applicable law or treaty (including future 89 | time extensions), (iii) in any current or future medium and for any number 90 | of copies, and (iv) for any purpose whatsoever, including without 91 | limitation commercial, advertising or promotional purposes (the 92 | "License"). The License shall be deemed effective as of the date CC0 was 93 | applied by Affirmer to the Work. Should any part of the License for any 94 | reason be judged legally invalid or ineffective under applicable law, such 95 | partial invalidity or ineffectiveness shall not invalidate the remainder 96 | of the License, and in such case Affirmer hereby affirms that he or she 97 | will not (i) exercise any of his or her remaining Copyright and Related 98 | Rights in the Work or (ii) assert any associated claims and causes of 99 | action with respect to the Work, in either case contrary to Affirmer's 100 | express Statement of Purpose. 101 | 102 | 4. Limitations and Disclaimers. 103 | 104 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 105 | surrendered, licensed or otherwise affected by this document. 106 | b. Affirmer offers the Work as-is and makes no representations or 107 | warranties of any kind concerning the Work, express, implied, 108 | statutory or otherwise, including without limitation warranties of 109 | title, merchantability, fitness for a particular purpose, non 110 | infringement, or the absence of latent or other defects, accuracy, or 111 | the present or absence of errors, whether or not discoverable, all to 112 | the greatest extent permissible under applicable law. 113 | c. Affirmer disclaims responsibility for clearing rights of other persons 114 | that may apply to the Work or any use thereof, including without 115 | limitation any person's Copyright and Related Rights in the Work. 116 | Further, Affirmer disclaims responsibility for obtaining any necessary 117 | consents, permissions or other rights required for any use of the 118 | Work. 119 | d. Affirmer understands and acknowledges that Creative Commons is not a 120 | party to this document and has no duty or obligation with respect to 121 | this CC0 or use of the Work. 122 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Awesome Active Learning [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) 2 | 3 | 🤩 A curated list of awesome Active Learning ! 🤩 4 | 5 | Background 6 | 7 | ![image](./fig/an_illustrative_AL_example.jpg) 8 | 9 | (`An illustrative example of pool-based active learning`. image source: [Settles, Burr](https://minds.wisconsin.edu/handle/1793/60660)) 10 | 11 | What is Active Learning? 12 | 13 | Active learning is a special case of machine learning in which a learning algorithm can interactively query a oracle (or some other information source) to label new data points with the desired outputs. 14 | 15 | ![image](./fig/active_learning_cycle.jpg) 16 | 17 | (`The pool-based active learning cycle`. image source: [Settles, Burr](https://minds.wisconsin.edu/handle/1793/60660)) 18 | 19 | There are situations in which unlabeled data is abundant but manual labeling is expensive. In such a scenario, learning algorithms can actively query the oracle for labels. This type of iterative supervised learning is called active learning. Since the learner chooses the examples, the number of examples to learn a concept can often be much lower than the number required in normal supervised learning. With this approach, there is a risk that the algorithm is overwhelmed by uninformative examples. Recent developments are dedicated to multi-label active learning, hybrid active learning and active learning in a single-pass (on-line) context, combining concepts from the field of machine learning (e.g. conflict and ignorance) with adaptive, incremental learning policies in the field of online machine learning. 20 | 21 | (source: [Wikipedia](https://en.wikipedia.org/wiki/Active_learning_(machine_learning))) 22 | 23 | Contributing 24 | 25 | If you find the awesome paper/code/book/tutorial or have some suggestions, please feel free to [pull requests](https://github.com/baifanxxx/awesome-active-learning/pulls) or contact or to add papers using the following Markdown format: 26 | 27 | ``` txt 28 | Year | Paper Name | Conference | [Paper](link) | [Code](link) | Tags | Notes | 29 | ``` 30 | 31 | Tags 32 | 33 | `Sur.`: survey | `Cri.`: critics | 34 | `Pool.`: pool-based sampling | `Str.`: stream-based sampling | `Syn.`: membership query synthesize | 35 | `Semi.`: semi-supervised learning | `Self.`: self-supervised learning | `RL.`: reinforcement learning | 36 | `FS.`: few-shot learning | `Meta.`: meta learning | 37 | 38 | Thanks for your valuable contribution to the research community. 😃 39 | 40 | --- 41 | 42 |
Table of Contents
43 | 44 | - [Books](#books) 45 | - [Surveys](#surveys) 46 | - [Papers](#papers) 47 | - [2024](#2024) 48 | - [2023](#2023) 49 | - [2022](#2022) 50 | - [2021](#2021) 51 | - [2020](#2020) 52 | - [2019](#2019) 53 | - [2018](#2018) 54 | - [2017](#2017) 55 | - [Before 2017](#before-2017) 56 | 57 | - [Turtorials](#turtorials) 58 | - [Tools](#tools) 59 | 60 | --- 61 | 62 | # Books 63 | 64 | * [Chapter 22 Active Learning: A Survey. from Data Classification: Algorithms and Applications](http://charuaggarwal.net/active-survey.pdf) Charu C. Aggarwa et al.(CRC Press, 2014) 65 | * [Active Learning](https://www.morganclaypool.com/doi/abs/10.2200/S00429ED1V01Y201207AIM018). Burr Settles. (CMU, 2012) 66 | 67 | # Surveys 68 | 69 | | Year | Paper | Author | Publication | Code | Notes | 70 | | --- | --- | :---: | :---: | --- | --- | 71 | | 2022 | [A Comparative Survey of Deep Active Learning](https://arxiv.org/pdf/2203.13450.pdf) | Xueying Zhan et al. | arXiv | [code](https://github.com/SineZHAN/deepALplus) | | 72 | | 2021 | [A Survey on Active Deep Learning: From Model-driven to Data-driven](https://arxiv.org/abs/2101.09933) | Peng Liu et al. | CSUR | | | 73 | | 2020 | [A Survey of Active Learning for Text Classification using Deep Neural Networks](https://arxiv.org/abs/2008.07267) | Christopher Schröder et al. | arXiv || | 74 | | 2020 | [A Survey of Deep Active Learning](https://arxiv.org/abs/2009.00236) | Pengzhen Ren et al. | CSUR | | | 75 | | 2009 | [Active Learning Literature Survey](https://minds.wisconsin.edu/handle/1793/60660) | Settles, Burr. | University of Wisconsin-Madison Department of Computer Sciences | | | 76 | 77 | 78 | 79 | # Papers 80 | 81 | ## 2024 82 | 83 | | Title | Publication | Paper | Code | Tags | Notes | 84 | | -------- | :-----: | :----: | :----: |----|----| 85 | |Active Prompt Learning in Vision Language Models|CVPR2024|[Paper](https://arxiv.org/abs/2311.11178)|[Code](https://github.com/kaist-dmlab/pcb)| `Pool.`, `FS.`| AL for Vision-Language Model | 86 | |Active Generalized Category Discovery|CVPR 2024|[Paper](https://arxiv.org/abs/2403.04272)|[Code](https://github.com/mashijie1028/ActiveGCD) | `Pool.`| More generalized AL considering unseen novel categories | 87 | |Plug and Play Active Learning for Object Detection|CVPR 2024|[Paper](https://arxiv.org/abs/2211.11612)|[Code](https://github.com/ChenhongyiYang/PPAL) | `Pool.`| AL for Object Detection | 88 | |Entropic Open-Set Active Learning|AAAI 2024|[Paper](https://arxiv.org/abs/2312.14126)|[Code](https://github.com/bardisafa/EOAL) | `Pool.`| Open-world AL | 89 | 90 | 91 | ## 2023 92 | 93 | | Title | Publication | Paper | Code | Tags | Notes | 94 | | -------- | :-----: | :----: | :----: |----|----| 95 | |Compute-Efficient Active Learning|NeurIPS 2023 Workshop ReALML|[Paper](https://openreview.net/pdf?id=G6ujG6LaKV)|[Code](https://github.com/aimotive/Compute-Efficient-Active-Learning) | `Pool.`, `Syn.` | Method-agnostic framework | 96 | 97 | ## 2022 98 | 99 | | Title | Publication | Paper | Code | Tags | Notes | 100 | | -------- | :-----: | :----: | :----: |----|----| 101 | |Active Learning Helps Pretrained Models Learn the Intended Task|NeurIPS|[paper](https://arxiv.org/abs/2204.08491)|[code](https://github.com/alextamkin/active-learning-pretrained-models)|`Pool.`|| 102 | |Making Your First Choice: To Address Cold Start Problem in Vision Active Learning|NeurIPS workshop|[paper](https://arxiv.org/abs/2210.02442)|[code](https://github.com/c-liangyu/CSVAL)|`Pool.`| Cold-start problem| 103 | |Active Learning Through a Covering Lens|NeurIPS|[paper](https://arxiv.org/abs/2205.11320)|[code](https://github.com/avihu111/TypiClust)|`Pool.`| | 104 | |Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation|NeurIPS|[paper](https://arxiv.org/pdf/2202.06881)|[code](https://github.com/jlko/active-surrogate-estimators)|`Pool.`|Model evaluation| 105 | |Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learning|NeurIPS|[paper](https://arxiv.org/pdf/2210.07805.pdf)|[code](https://github.com/kaist-dmlab/MQNet)|`Pool.`|| 106 | |One-Bit Active Query With Contrastive Pairs|CVPR|[paper](https://openaccess.thecvf.com/content/CVPR2022/html/Zhang_One-Bit_Active_Query_With_Contrastive_Pairs_CVPR_2022_paper.html)| |`Pool.`|One-bit supervision task| 107 | |Active label cleaning for improved dataset quality under resource constraints |Nature Communications|[paper](https://arxiv.org/abs/2109.00574)|[code](https://github.com/microsoft/InnerEye-DeepLearning/tree/1606729c7a16e1bfeb269694314212b6e2737939/InnerEye-DataQuality)|`Pool.`|Label cleaning| 108 | |Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation |CVPR|[paper](https://arxiv.org/abs/2111.12940)|[code](https://github.com/BIT-DA/RIPU)|`Pool.`|| 109 | |Budget-aware Few-shot Learning via Graph Convolutional Network |arXiv|[paper](https://arxiv.org/abs/2201.02304)||`Pool.` `Meta.` `FS.`|| 110 | |Using Self-Supervised Pretext Tasks for Active Learning |arXiv|[paper](https://arxiv.org/abs/2201.07459)|[code](https://github.com/johnsk95/PT4AL)|`Pool.` `SS.`|Cold-start problem| 111 | |Low-Budget Active Learning via Wasserstein Distance: An Integer Programming Approach|ICLR|[paper](https://arxiv.org/abs/2106.02968)||`Pool.`| Cold-start problem| 112 | |Active Learning by Feature Mixing|CVPR|[paper](https://arxiv.org/abs/2203.07034)|[code](https://github.com/AminParvaneh/alpha_mix_active_learning)|`Pool.`|| 113 | |ALLSH: Active Learning Guided by Local Sensitivity and Hardness|NAACL|[paper](https://arxiv.org/abs/2205.04980)|[code](https://github.com/szhang42/allsh)|`Semi.`|NLP| 114 | |Coherence-based Label Propagation over Time Series for Accelerated Active Learning|ICLR|[paper](https://openreview.net/forum?id=gjNcH0hj0LM)| [code](https://github.com/kaist-dmlab/TCLP) |`Pool.`|Time series| 115 | 116 | ## 2021 117 | 118 | | Title | Publication | Paper | Code | Tags | Notes | 119 | | -------- | :-----: | :----: | :----: |----|----| 120 | | Active learning with MaskAL reduces annotation effort for training Mask R-CNN | arXiv | [paper](https://arxiv.org/abs/2112.06586) | [code](https://github.com/pieterblok/maskal) | | | 121 | | MedSelect: Selective Labeling for Medical Image Classification Combining Meta-Learning with Deep Reinforcement Learning |arXiv|[paper](https://arxiv.org/abs/2103.14339)| [code](https://github.com/stanfordmlgroup/MedSelect) |`Pool.` `Meta.` `RL.`| | 122 | | Can Active Learning Preemptively Mitigate Fairness Issues |ICLR-RAI|[paper](https://arxiv.org/abs/2104.06879)|[code](https://github.com/ElementAI/active-fairness)|`Pool.`|Thinking fairness issues| 123 | |Sequential Graph Convolutional Network for Active Learning |CVPR|[paper](https://arxiv.org/pdf/2006.10219.pdf)|[code](https://github.com/razvancaramalau/Sequential-GCN-for-Active-Learning)|`Pool.`| | 124 | |Task-Aware Variational Adversarial Active Learning |CVPR|[paper](https://arxiv.org/abs/2002.04709)|[code](https://github.com/cubeyoung/TA-VAAL)|`Pool.`| | 125 | |Effective Evaluation of Deep Active Learning on Image Classification Tasks |arXiv|[paper](https://arxiv.org/abs/2106.15324)||`Cri.`| | 126 | |Semi-Supervised Active Learning for Semi-Supervised Models: Exploit Adversarial Examples With Graph-Based Virtual Labels |ICCV|[paper](https://openaccess.thecvf.com/content/ICCV2021/papers/Guo_Semi-Supervised_Active_Learning_for_Semi-Supervised_Models_Exploit_Adversarial_Examples_With_ICCV_2021_paper.pdf)||`Pool.` `Semi.`| | 127 | |Contrastive Coding for Active Learning under Class Distribution Mismatch |ICCV|[paper](https://openaccess.thecvf.com/content/ICCV2021/html/Du_Contrastive_Coding_for_Active_Learning_Under_Class_Distribution_Mismatch_ICCV_2021_paper.html)|[code](https://github.com/RUC-DWBI-ML/CCAL)|`Pool.`|Defines a good question| 128 | |Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering |ACL-IJCNLP|[paper](https://arxiv.org/abs/2107.02331)|[code](https://github.com/siddk/vqa-outliers)|`Pool.`|Thinking about outliers | 129 | |LADA: Look-Ahead Data Acquisition via Augmentation for Active Learning |NeurIPS|[paper](https://arxiv.org/abs/2011.04194)||`Pool.`| | 130 | |Multi-Anchor Active Domain Adaptation for Semantic Segmentation |ICCV|[paper](https://arxiv.org/abs/2108.08012)|[code](https://github.com/munanning/mada)|`Pool.`| | 131 | |Active Learning for Lane Detection: A Knowledge Distillation Approach |ICCV|[paper](https://openaccess.thecvf.com/content/ICCV2021/html/Peng_Active_Learning_for_Lane_Detection_A_Knowledge_Distillation_Approach_ICCV_2021_paper.html)||`Pool.`| | 132 | |Active Contrastive Learning of Audio-Visual Video Representations |ICLR|[paper](https://arxiv.org/abs/2009.09805)|[code](https://github.com/yunyikristy/CM-ACC)|`Pool.`| | 133 | |Multiple instance active learning for object detection |CVPR|[paper](https://arxiv.org/abs/2104.02324)|[code](https://github.com/yuantn/MI-AOD)|`Pool.`| | 134 | |SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency |NeurIPS|[paper](https://arxiv.org/abs/2112.01001)||`Self.`|Robot exploration| 135 | |Influence Selection for Active Learning |ICCV|[paper](https://openaccess.thecvf.com/content/ICCV2021/html/Liu_Influence_Selection_for_Active_Learning_ICCV_2021_paper.html)|[code](https://github.com/dragonlzm/ISAL)|`Pool.`|| 136 | |Reducing Label Effort: Self-Supervised meets Active Learning |arXiv|[paper](https://arxiv.org/abs/2108.11458)||`Pool.` `Self.` `Cri.`| A meaningful attempt on the combination of SS & AL| 137 | |Towards General and Efficient Active Learning |arXiv|[paper](https://arxiv.org/abs/2112.07963)|[code](https://github.com/yichen928/GEAL_active_learning)|`Pool.` `Self.`| Single-pass AL based on SS ViT| 138 | |Cartography Active Learning |EMNLP Findings|[paper](https://arxiv.org/abs/2109.04282)|[code](https://github.com/jjzha/cal)|`Pool.`| | 139 | |Joint Semi-supervised and Active Learning for Segmentation of Gigapixel Pathology Images with Cost-Effective Labeling |ICCVW|[paper](https://openaccess.thecvf.com/content/ICCV2021W/CDPath/papers/Lai_Joint_Semi-Supervised_and_Active_Learning_for_Segmentation_of_Gigapixel_Pathology_ICCVW_2021_paper.pdf)||`Pool.`| | 140 | |PAL : Pretext-based Active Learning|BMVC|[paper](https://arxiv.org/abs/2010.15947)|[code](https://github.com/shubhangb97/PAL_pretext_based_active_learning)|`Pool.`| Cold-start problem| 141 | |Active Learning for Deep Object Detection via Probabilistic Modeling|ICCV|[paper](https://openaccess.thecvf.com/content/ICCV2021/html/Choi_Active_Learning_for_Deep_Object_Detection_via_Probabilistic_Modeling_ICCV_2021_paper.html)|[code](https://github.com/NVlabs/AL-MDN)|`Pool.`|GMM| 142 | |Unsupervised Data Selection for Data-Centric Semi-Supervised Learning|arXiv|[paper](https://arxiv.org/abs/2110.03006)||`Pool.`|Data selection + SSL| 143 | |Batch Active Learning at Scale|NeurIPS|[paper](https://arxiv.org/abs/2107.14263)||`Scale.` `Pool.`| | 144 | 145 | 146 | ## 2020 147 | 148 | | Title | Publication | Paper | Code | Tags | Notes | 149 | | -------- | :-----: | :----: | :----: |----|----| 150 | | Contextual Diversity for Active Learning | ECCV | [paper](https://link.springer.com/chapter/10.1007/978-3-030-58517-4_9) | [code](https://github.com/sharat29ag/CDAL) | `Pool.`| | 151 | | Active Learning for BERT: An Empirical Study | EMNLP | [paper](https://aclanthology.org/2020.emnlp-main.638/) | [code](https://github.com/IBM/low-resource-text-classification-framework) |`Pool.`| | 152 | | Reinforced active learning for image segmentation |ICLR|[paper](https://arxiv.org/abs/2002.06583)|[code](https://github.com/ArantxaCasanova/ralis)|`Pool.` `RL.`| | 153 | |Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds|ICLR|[paper](https://arxiv.org/abs/1906.03671)|[code](https://github.com/JordanAsh/badge)|`Pool.`| | 154 | |Adversarial Sampling for Active Learning|WACV|[paper](https://openaccess.thecvf.com/content_WACV_2020/html/Mayer_Adversarial_Sampling_for_Active_Learning_WACV_2020_paper.html)||`Pool.`| | 155 | |Online Active Learning of Reject Option Classifiers|AAAI|[paper](https://ojs.aaai.org/index.php/AAAI/article/view/6019/5875)||| | 156 | | ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation |CVPR|[paper](https://openaccess.thecvf.com/content_CVPR_2020/html/Siddiqui_ViewAL_Active_Learning_With_Viewpoint_Entropy_for_Semantic_Segmentation_CVPR_2020_paper.html)| [code](https://github.com/nihalsid/ViewAL) |`Pool.`| | 157 | |Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision|CVPR|[paper](http://openaccess.thecvf.com/content_CVPR_2020/papers/Gudovskiy_Deep_Active_Learning_for_Biased_Datasets_via_Fisher_Kernel_Self-Supervision_CVPR_2020_paper.pdf)| [code](https://github.com/gudovskiy/al-fk-self-supervision) || | 158 | | Deep Reinforcement Active Learning for Medical Image Classification |MICCAI|[paper](https://link.springer.com/chapter/10.1007/978-3-030-59710-8_4)||`Pool.` `RL.`| | 159 | | State-Relabeling Adversarial Active Learning |CVPR|[paper](https://openaccess.thecvf.com/content_CVPR_2020/html/Zhang_State-Relabeling_Adversarial_Active_Learning_CVPR_2020_paper.html)|[code](https://github.com/Beichen1996/SRAAL)|`Pool.` | | 160 | |Towards Robust and Reproducible Active Learning Using Neural Networks|arXiv|[paper](https://arxiv.org/pdf/2002.09564)| [code](https://github.com/acl21/deep-active-learning-pytorch) |`Cri.`| | 161 | | Minimax Active Learning | arXiv |[paper](https://arxiv.org/pdf/2012.10467v2.pdf)||| | 162 | | Bayesian Force Fields from Active Learning for Simulation of Inter-Dimensional Transformation of Stanene | npj Computational Materials | [paper](https://arxiv.org/pdf/2008.11796v2.pdf) | [code](https://github.com/mir-group/flare) || | 163 | |Consistency-Based Semi-supervised Active Learning: Towards Minimizing Labeling Cost|ECCV|[paper](https://link.springer.com/chapter/10.1007/978-3-030-58607-2_30)||`Pool.` `Semi.`| | 164 | |Cold-start Active Learning through Self-supervised Language Modeling|EMNLP|[paper](https://arxiv.org/abs/2010.09535)| [code](https://github.com/forest-snow/alps) |`Pool.` `SS.`|| 165 | 166 | 167 | ## 2019 168 | 169 | | Title | Publication | Paper | Code | Tags | Notes | 170 | | -------- | :-----: | :----: | :----: |----|----| 171 | | Generative Adversarial Active Learning for Unsupervised Outlier Detection | TKDE | [paper](https://arxiv.org/pdf/1809.10816v4.pdf) | [code](https://github.com/leibinghe/GAAL-based-outlier-detection) ||| 172 | | Bayesian Generative Active Deep Learning |ICML|[paper](http://proceedings.mlr.press/v97/tran19a.html)|[code](https://github.com/toantm/BGADL)|`Pool.` `Semi.`| | 173 | | Variational Adversarial Active Learning |ICCV|[paper](https://openaccess.thecvf.com/content_ICCV_2019/html/Sinha_Variational_Adversarial_Active_Learning_ICCV_2019_paper.html)|[code](https://github.com/sinhasam/vaal)|`Pool.` `Semi.`| | 174 | |Integrating Bayesian and Discriminative Sparse Kernel Machines for Multi-class Active Learning|NeurIPS|[paper](https://papers.nips.cc/paper/2019/hash/bcc0d400288793e8bdcd7c19a8ac0c2b-Abstract.html)|| | | 175 | |Active Learning via Membership Query Synthesisfor Semi-supervised Sentence Classification|CoNLL|[paper](https://www.aclweb.org/anthology/K19-1044/)|| | | 176 | |Discriminative Active Learning|arXiv|[paper](https://arxiv.org/pdf/1907.06347.pdf)| [code](https://github.com/dsgissin/DiscriminativeActiveLearning) | | | 177 | |Semantic Redundancies in Image-Classification Datasets: The 10% You Don’t Need|arXiv|[paper](https://arxiv.org/pdf/1901.11409.pdf)|| | | 178 | |On-the-Fly Bayesian Active Learning of Interpretable Force-Fields for Atomistic Rare Events|npj Computational Materials|[paper](https://arxiv.org/pdf/1904.02042v2.pdf)| [code](https://github.com/mir-group/flare) | | | 179 | |Bayesian Batch Active Learning as Sparse Subset Approximation|NIPS|[paper](http://papers.nips.cc/paper/8865-bayesian-batch-active-learning-as-sparse-subset-approximation.pdf)| [code](https://github.com/rpinsler/active-bayesian-coresets) | | | 180 | | Learning Loss for Active Learning |CVPR|[paper](https://openaccess.thecvf.com/content_CVPR_2019/html/Yoo_Learning_Loss_for_Active_Learning_CVPR_2019_paper.html)|[code](https://github.com/Mephisto405/Learning-Loss-for-Active-Learning)|`Pool.` | | 181 | |Rapid Performance Gain through Active Model Reuse|IJCAI|[paper](http://www.lamda.nju.edu.cn/liyf/paper/ijcai19-acmr.pdf)|| | | 182 | |Parting with Illusions about Deep Active Learning|arXiv|[paper](https://arxiv.org/abs/1912.05361)||`Cri.` | | 183 | |BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning|NIPS|[paper](http://papers.nips.cc/paper/8925-batchbald-efficient-and-diverse-batch-acquisition-for-deep-bayesian-active-learning.pdf)| [code](https://github.com/BlackHC/BatchBALD) | | | 184 | 185 | 186 | ## 2018 187 | 188 | | Title | Publication | Paper | Code | Tags | Notes | 189 | | -------- | :-----: | :----: | :----: |----|----| 190 | |The Power of Ensembles for Active Learning in Image Classification|CVPR|[paper](http://openaccess.thecvf.com/content_cvpr_2018/papers/Beluch_The_Power_of_CVPR_2018_paper.pdf)|||| 191 | | Adversarial Learning for Semi-Supervised Semantic Segmentation |BMVC|[paper](https://arxiv.org/abs/1802.07934)|[code](https://github.com/hfslyc/AdvSemiSeg)|`Pool.` `Semi.`| | 192 | |A Variance Maximization Criterion for Active Learning|Pattern Recognition|[paper](https://www.sciencedirect.com/science/article/pii/S0031320318300256)| [code](https://github.com/YazhouTUD/MVAL) ||| 193 | |Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning|ICLR-WS|[paper](https://arxiv.org/abs/1806.04798)||`Pool.` `Meta.` `RL.`|| 194 | |Active Learning for Convolutional Neural Networks: A Core-Set Approach|ICLR|[paper](https://openreview.net/pdf?id=H1aIuk-RW)|||| 195 | |Adversarial Active Learning for Sequence Labeling and Generation|IJCAI|[paper](https://www.ijcai.org/proceedings/2018/0558.pdf)|||| 196 | |Meta-Learning for Batch Mode Active Learning|ICLR-WS|[paper](https://openreview.net/references/pdf?id=r1PsGFJPz)|||| 197 | |Adversarial Active Learning for Deep Networks: a Margin Based Approach | ICML | [paper](https://arxiv.org/abs/1802.09841) | | | | 198 | |CEREALS - Cost-Effective REgion-based Active Learning for Semantic Segmentation | BMVC | [paper](https://arxiv.org/abs/1810.09726) | | | | 199 | 200 | ## 2017 201 | 202 | | Title | Publication | Paper | Code | Tags | Notes | 203 | | -------- | :-----: | :----: | :----: |----|----| 204 | |Active Decision Boundary Annotation with Deep Generative Models|ICCV|[paper](http://openaccess.thecvf.com/content_ICCV_2017/papers/Huijser_Active_Decision_Boundary_ICCV_2017_paper.pdf)| [code](https://github.com/MiriamHu/ActiveBoundary) ||| 205 | | Active One-shot Learning |CoRR|[paper](https://arxiv.org/abs/1702.06559)|[code](https://github.com/markpwoodward/active_osl)|`Str.` `RL.` `FS.`| | 206 | | A Meta-Learning Approach to One-Step Active-Learning |AutoML@PKDD/ECML|[paper](https://arxiv.org/abs/1706.08334)||`Pool.` `Meta.`| | 207 | | Generative Adversarial Active Learning |arXiv|[paper](https://arxiv.org/abs/1702.07956)||`Pool.` `Syn.`| | 208 | |Active Learning from Peers|NIPS|[paper](http://papers.neurips.cc/paper/7276-active-learning-from-peers.pdf)|||| 209 | |Learning Active Learning from Data|NIPS|[paper](https://arxiv.org/abs/1703.03365)|[code](https://github.com/ksenia-konyushkova/LAL)|`Pool.`|| 210 | |Learning Algorithms for Active Learning|ICML|[paper](http://proceedings.mlr.press/v70/bachman17a.html)|||| 211 | | Deep Bayesian Active Learning with Image Data|ICML|[paper](http://proceedings.mlr.press/v70/gal17a)|[code](https://github.com/Riashat/Active-Learning-Bayesian-Convolutional-Neural-Networks/tree/master/ConvNets/FINAL_Averaged_Experiments/Final_Experiments_Run)|`Pool.` | | 212 | |Learning how to Active Learn: A Deep Reinforcement Learning Approach|EMNLP|[paper](https://arxiv.org/abs/1708.02383)|[code](https://github.com/mengf1/PAL)|`Str.` `RL.`|| 213 | 214 | ## Before 2017 215 | 216 | |Year| Title | Publication | Paper | Code | Tags | Notes | 217 | |----| -------- | :-----: | :----: | :----: |----|----| 218 | |2016|Active Image Segmentation Propagation|CVPR|[paper](http://openaccess.thecvf.com/content_cvpr_2016/papers/Jain_Active_Image_Segmentation_CVPR_2016_paper.pdf)|| | | 219 | |2016|Cost-Effective Active Learning for Deep Image Classification|TCSVT|[paper](https://arxiv.org/pdf/1701.03551.pdf)| [code](https://github.com/dhaalves/CEAL_keras) | | | 220 | |2015|Multi-Label Active Learning from Crowds| arXiv |[paper](https://arxiv.org/pdf/1508.00722v1.pdf)|| | | 221 | |2015|Active Learning by Learning | AAAI | [paper](https://ojs.aaai.org/index.php/AAAI/article/view/9597) | | | | 222 | |2014|Beyond Disagreement-based Agnostic Active Learning|NIPS|[paper](https://arxiv.org/abs/1407.2657)|| | | 223 | |2014|Active Semi-Supervised Learning Using Sampling Theory for Graph Signals|KDD|[paper](https://dl.acm.org/doi/abs/10.1145/2623330.2623760)| [code](https://github.com/broshanfekr/Active_Semi-Supervised_Learning_Using_Sampling_Theory_for_Graph_signals) | | | 224 | |2013|Active Learning for Probabilistic Hypotheses Usingthe Maximum Gibbs Error Criterion|NIPS|[paper](https://eprints.qut.edu.au/114032/)|| | | 225 | |2013|Active Learning for Multi-Objective Optimization|ICML|[paper](http://proceedings.mlr.press/v28/zuluaga13.html)|| | | 226 | |2012|Batch Active Learning via Coordinated Matching|ICML|[paper](https://arxiv.org/abs/1206.6458)|| | | 227 | |2012|Bayesian Optimal Active Search and Surveying|ICML|[paper](https://arxiv.org/abs/1206.6406)| [code](https://github.com/rmgarnett/active_search)| | | 228 | |2011|Active Learning Using On-line Algorithms|KDD|[paper](https://dl.acm.org/doi/abs/10.1145/2020408.2020553)|| | | 229 | |2011|Bayesian Active Learning for Classification and Preference Learning|CoRR|[paper](https://arxiv.org/abs/1112.5745)| [code](https://github.com/cambridge-mlg/BALaudiogram)| | | 230 | |2011|Active Learning from Crowds|ICML|[paper](https://openreview.net/pdf?id=yVemp8x6Av3y)|| | | 231 | |2011|Ask Me Better Questions: Active Learning Queries Based on Rule Induction|KDD|[paper](https://dl.acm.org/doi/abs/10.1145/2020408.2020559)|| | | 232 | |2010|Active Instance Sampling via Matrix Partition|NIPS|[paper](http://people.scs.carleton.ca/~yuhongguo/research/papers/activenips10figs.pdf)|| | | 233 | |2008|Hierarchical Sampling for Active Learning|ICML|[paper](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.8661&rep=rep1&type=pdf)|| | | 234 | |2008|An Analysis of Active Learning Strategies for Sequence Labeling Tasks|EMNLP|[paper](https://www.aclweb.org/anthology/D08-1112.pdf)|| | | 235 | |2008|Active Learning with Direct Query Construction|KDD|[paper](https://dl.acm.org/doi/abs/10.1145/1401890.1401950)|| | | 236 | |2007|Discriminative Batch Mode Active Learning|NIPS|[paper](https://dl.acm.org/doi/pdf/10.1145/1390156.1390183)| [code](https://github.com/dsgissin/DiscriminativeActiveLearning) | | | 237 | |1994|Improving Generalization with Active Learning|Machine Learning|[paper](https://link.springer.com/content/pdf/10.1007/BF00993277.pdf)|| | | 238 | 239 | # Turtorials 240 | 241 | * [Overview of Active Learning for Deep Learning](https://jacobgil.github.io/deeplearning/activelearning). Jacob Gildenblat. 242 | * [Active Learning from Theory to Practice](https://www.youtube.com/watch?v=_Ql5vfOPxZU). Steve Hanneke, Robert Nowak. (ICML, 2019) 243 | 244 | # Tools 245 | * [Active-Learning-as-a-Service: An Efficient MLOps System for Data-Centric AI](https://github.com/MLSysOps/Active-Learning-as-a-Service). Huang, Yizheng and Zhang, Huaizheng and Li, Yuanming and Lau, Chiew Tong and You, Yang. (2022) 246 | * [[BAAL] A Bayesian Active Learning Library](https://github.com/ElementAI/baal/). Atighehchian, Parmida and Branchaud-Charron, Frederic and Freyberg, Jan and Pardinas, Rafael and Schell, Lorne. (2019) 247 | * [ALiPy: Active Learning in Python](https://github.com/NUAA-AL/alipy). Ying-Peng Tang, Guo-Xiang Li, Sheng-Jun Huang. (NUAA, 2019) 248 | * [modAL: A modular active learning framework for Python](https://github.com/modAL-python/modAL). Tivadar Danka and Peter Horvath. (2018) 249 | 250 | 251 | --------------------------------------------------------------------------------